2.1 Unadjusted network meta-regression (NMR)

We synthesize the evidence from RCT and NRS without acknowledging the differences between them. We combine the IPD data from RCT and NRS in one model and we do the same in another model with the AD information. Then, we combine the estimates from both parts as described in Section 2.5.

NMR model for IPD studies

\[ y_{ijk} \sim Bernoulli(p_{ijk}) \] \[\begin{equation} logit(p_{ijk}) = \begin{cases} u_{jb} +\beta_{0j} x_{ijk} & \text{if $k=b$}\\ u_{jb} +\delta_{jbk} + \beta_{0j}x_{ijk}+\beta^w_{1,jbk}x_{ijk} + (\beta^B_{1,jbk}-\beta^w_{1,jbk}) \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\]

NMR model for AD studies \[ r_{jk} \sim Binomial(p_{.jk},n_{jk}) \] \[\begin{equation} logit(p_{.jk}) = \begin{cases} u_{jb} & \text{if $k=b$}\\ u_{jb} +\delta_{jbk} +\beta^B_{1,jbk} \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\]

2.2 Using non-randomized studies (NRS) to construct priors for the treatment effects

First, the (network) meta-regression with only NRS data estimates the relative treatment effects with posterior distribution of mean \(\tilde{d}^{NRS}_{Ak}\) and variance \(V^{NRS}_{Ak}\) (use run.nrs in crossnma.model() to control this process). The posteriors of NRS results are then used as priors for the corresponding basic parameters in the RCT model, \(d_{Ak} \sim \mathcal{N}(\tilde{d}^{NRS}_{Ak},V^{NRS}_{Ak})\). We can adjust for potential biases associated with NRS by either shifting the mean of the prior distribution with a bias term \(\zeta\) or by dividing the prior variance with a common inflation factor \(w, 0<w<1\) controls NRS contribution. The assigned priors become \(d_{Ak} \sim \mathcal{N}(\tilde{d}^{NRS}_{Ak}+\zeta,V^{NRS}_{Ak}/w)\).

2.3 Bias-adjusted model 1

We incorporate judgments about study risk of bias (RoB) in bias-adjusted model 1 and model 2. Each judgment about the risk of bias in a study is summarized by the index \(R_j\) which takes binary values 0 (no bias) or 1 (bias). In bias-adjusted model 1, we extend the method introduced by Dias et al. (2010) by adding a treatment-specific bias term \(\gamma_{2,jbk} R_j\) to the relative treatment effect on both the AD and IPD parts of the model. A multiplicative model can also be employed, where treatment effects are multiplied by \(\gamma_{1,jbk}^{R_j}\). We can add either multiplicative bias effects, additive bias effects, or both (in this case, \(\delta_{jbk}\) should be dropped from the additive part). The models in previous section are extended to adjust for bias as follows.

NMR model for IPD studies

\[\begin{equation} logit(p_{ijk}) = \begin{cases} u_{jb} +\beta_{0j} x_{ijk} & \text{if $k=b$}\\ u_{jb} +\overbrace{\delta_{jbk} \gamma_{1,jbk}^{R_j}}^{\text{multiplicative}}+\overbrace{\delta_{jbk}+\gamma_{2,jbk} R_j}^{\text{additive}}+ \beta_{0j}x_{ijk}+\beta^w_{1,jbk} x_{ijk}+ (\beta^B_{1,jbk}-\beta^w_{1,jbk}) \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\]

NMR model for AD studies

\[\begin{equation} logit(p_{.jk}) = \begin{cases} u_{jb} & \text{if $k=b$}\\ u_{jb} +\overbrace{\delta_{jbk} \gamma_{1,jbk}^{R_j}}^{\text{multiplicative}}+\overbrace{\delta_{jbk}+\gamma_{2,jbk} R_j}^{\text{additive}}+ \beta^B_{1,jbk} \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\]

The bias indicator \(R_j\) follows the following distribution

\[ R_j \sim Bernoulli(\pi_j) \] The bias probabilities \(\pi_j\) are study-specific and can be estimated in two different ways. They are either given informative beta priors (\({Beta(a_1,a_2)}\)) that are set according to the risk of bias for each study. \[ \pi_j \sim Beta(a_1, a_2) \] The hyperparameters \(a_1\) and \(a_2\) should be chosen in a way that reflects the risk of bias for each study. The degree of skewness in beta distribution can be controlled by the ratio \(a_1/a_2\) . When \(a_1/a_2\) equals 1 (or \(a_1=a_2\)), there is no skewness in the beta distribution (the distribution is reduced to a uniform distribution), which is appropriate for studies with unclear risk of bias. When the ratio \(a_1/a_2\) is closer to 1, the more the mean of probability of bias (expected value of \(\pi_j=a_1/(a_1+a_2))\) gets closer to 1 and the study acquires ‘major’ bias adjustment. The default beta priors are as follows: high bias RCT pi.high.rct='dbeta(10,1)', low bias RCT pi.low.rct='dbeta(1,10)', high bias NRS pi.high.nrs='dbeta(30,1)' and low bias NRS pi.low.nrs='dbeta(1,30)'. Alternatively, we can use the study characteristics \(z_j\) to estimate \(\pi_j\) through a logistic transformation (internally coded).

We combine the multiplicative and the additive treatment-specific bias effects across studies by assuming they are exchangeable \(\gamma_{1,jbk}\sim \mathcal{N}(g_{1,bk},\tau_{1,\gamma}^2 )\),\(\gamma_{2,jbk}\sim \mathcal{N}(g_{2,bk},\tau_{2,\gamma}^2 )\)) or common \(\gamma_{1,jbk}=g_{1,bk}\) and \(\gamma_{2,jbk}=g_{2,bk}\). Dias et al. (2010) proposed to model the mean bias effect \((g_{1,bk}, g_{2,bk})\) based on the treatments being compared.
\[\begin{equation} g_{m,bk} = \begin{cases} g_m & \text{if $b$ is inactive treatment}\\ 0 \text{ or } (-1)^{dir_{bk}} g_m^{act} & \text{if $b$ and $k$ are active treatments} \end{cases} \end{equation}\] where \(m={1,2}\). This approach assumes a common mean bias for studies that compare active treatments with an inactive treatment (placebo, standard or no treatment). For active vs active comparisons, we could assume either a zero mean bias effect or a common bias effect \(g_m^{act}\). The direction of bias \(dir_{bk}\) in studies that compare active treatments with each other should be defined in the data. That is set to be either 0, meaning that bias favors \(b\) over \(k\), or 1 , meaning that \(k\) is favored to \(b\). In crossnma.model(), the bias direction is specified by providing the unfavoured treatment for each study, unfav. To select which mean bias effect should be applied, the user can provide the bias.group column as data. Its values can be 0 (no bias adjustment), 1 (to assign for the comparison mean bias effect \(g_m\)) or 2 (to set bias \(g_m^{act}\)).

Another parameterisation of the logistic model with additive bias effect is

NMR model for IPD studies

\[\begin{equation} logit(p_{ijk}) = \begin{cases} u_{jb} +\beta_{0j} x_{ijk} & \text{if $k=b$}\\ u_{jb} +(1-R_j)\delta_{jbk}+\delta_{jbk}^{bias}R_j+ \beta_{0j}x_{ijk}+\beta^w_{1,jbk} x_{ijk}+ (\beta^B_{1,jbk}-\beta^w_{1,jbk}) \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\]

NMR model for AD studies

\[\begin{equation} logit(p_{.jk}) = \begin{cases} u_{jb} & \text{if $k=b$}\\ u_{jb} +(1-R_j)\delta_{jbk}+\delta_{jbk}^{bias}R_j+ \beta^B_{1,jbk} \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\] Then the bias-adjusted relative treatment effect (\(\delta_{jbk}^{bias}=\delta_{jbk}+\gamma_{jbk}\)) can be assumed exchangeable across studies \(\delta_{jbk}^{bias} \sim \mathcal{N}(g_{bk}+d_{Ak}-d_{Ab}, \tau^2 /q_j)\) or fixed as \(\delta_{jbk}^{bias}=g_{bk} + d_{Ak}-d_{Ab}\). In this parameterisation, instead of assigning prior to the between-study heterogeneity in bias effect \(\tau_{\gamma}\), we model the RoB weight \(q_j=\tau^2/(\tau^2+\tau_{\gamma}^2\)) for each study. This quantity \(0<q_j<1\) quantifies the proportion of the between-study heterogeneity that is not explained by accounting for risk of bias. The values of \(v\) determine the extent studies at high risk of bias will be down-weighted on average. Setting \(v=1\) gives \(E(q_j )=v/(v+1)=0.5\), which means that high risk of bias studies will be penalized by 50% on average. In crossnma.model(), the user can assign the average down-weight \(E(q_j )\) to the argument down.wgt.

2.4 Bias-adjusted model 2

Another way to incorporate the RoB of the study is by replacing \(\delta_{jbk}\) by a “bias-adjusted” relative treatment effect \(\theta_{jbk}\). Then \(\theta_{jbk}\) is modeled with a bimodal normal distribution as described in Section 2.5. For more details see Verde (2020).

NMR model for IPD studies

\[\begin{equation} logit(p_{ijk}) = \begin{cases} u_{jb} +\beta_{0j} x_{ijk} & \text{if $k=b$}\\ u_{jb} +\theta_{jbk} + \beta_{0j} x_{ijk}+\beta^w_{1,jbk} x_{ijk}+ (\beta^B_{1,jbk}-\beta^w_{1,jbk}) \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\]

NMR model for AD studies

\[\begin{equation} logit(p_{jk}) = \begin{cases} u_{jb} & \text{if $k=b$}\\ u_{jb} +\theta_{jbk} +\beta^B_{1,jbk} \bar{x}_{j} & \text{if $k\ne b$} \end{cases} \end{equation}\]

where the bias-adjusted relative treatment effect (\(\theta_{jk}\)) are modeled via random-effects model with a mixture of two normal distributions. \[ \theta_{jbk} \sim (1-\pi_j) \mathcal{N}(d_{Ak}-d_{Ab}, \tau^2) + \pi_j \mathcal{N}(d_{Ak}-d_{Ab}+\gamma_{jbk}, \tau^2+\tau_\gamma^2) \]

Alternatively, we can summarize these relative effects assuming a common-effect model \[ \theta_{jbk}= d_{Ak}-d_{Ab}+\pi_j \gamma_{jbk} \]

2.5 Assumptions about the model parameters

The table below summarizes the different assumptions implemented in the package about combining the parameters in the models described above.

3.1 Description of the data

The data we use are fictitious but have been developed to resample to real RCTs with IPD and aggregate data included in Tramacere and Filippini (2015). The studies provide either aggregate data stddata (2 RCTs) or as individual participant data ipddata (3 RCTs and 1 cohort study). Both datasets compare in total four drugs which are anonymized.

The ipddata contains 2950 rows, each row refers to a participant in the study. We display the first few rows of the data set:

head(ipddata)
#>    id relapse treat design age sex rob unfavored bias.group year
#> 1   1       0     D    rct  22   0 low         1          1 2002
#> 3   1       0     D    rct  37   0 low         1          1 2002
#> 6   1       0     D    rct  34   1 low         1          1 2002
#> 7   1       0     D    rct  38   1 low         1          1 2002
#> 9   1       0     D    rct  31   0 low         1          1 2002
#> 10  1       0     D    rct  38   0 low         1          1 2002

For each participant, we have information for the outcome relapse (0=no, 1=yes), the treatment label treat, the age (in years) and sex (0 = Female, 1 = Male) of the participant. The following columns are set on study-level (it is repeated for each participant in each study): the id, the design of the study (needs to be either rct or nrs), the risk of bias rob on each study (can be set as low, high or unclear), the year of publication, the bias.group for the study comparison and the study unfavoured treatment unfavored.

The aggregate data has the standard format for meta-analysis with the exact same variable names

head(stddata)
#>   id relapse   n treat design  age sex  rob unfavored bias.group year
#> 1  1      19  25     A    rct 34.3 0.2 high         0          1 2010
#> 2  1      11  25     C    rct 34.3 0.3 high         1          1 2010
#> 3  2      97 126     A    rct 30.0 0.4 high         0          1 2015
#> 4  2      89 125     C    rct 30.0 0.5 high         1          1 2015

3.2 Analysis

There are two steps to run the NMA/NMR model. The first step is to create a JAGS model using crossnma.model() which produces the JAGS code and the data. In the second step, the output of that function will be used in crossnma() to run the analysis through JAGS.

# jags model: code+data
mod1 <- crossnma.model(treat, id, relapse, n, design,
  prt.data = ipddata, std.data = stddata,
  reference = NULL, trt.effect = "random",
  #---------- bias adjustment ----------
  method.bias = "naive",
  #---------- assign a prior ----------
  prior = list(tau.trt='dunif(0,3)')
  )
#> Both designs are combined naively without acknowledging design differences

netgraph(mod1)

# run jags
jagsfit1 <- crossnma(mod1,
  n.adapt = 500, n.iter = 5000, n.burnin = 2000,
  thin = 1, n.chains = 2)

knitr::kable(summary(jagsfit1, exp = FALSE), digits = 3)

oldpar <- par(mar = rep(2, 4), mfrow = c(2, 3))
plot(jagsfit1)

# jags model: code+data
mod2 <- crossnma.model(treat, id, relapse, n, design,
  prt.data = ipddata, std.data = stddata,
  reference = "A", trt.effect = "random",
  #---------- bias adjustment ----------
  method.bias = "naive",
  #----------  meta-regression ----------
  cov1 = age,
  split.regcoef = FALSE
  )
#> Both designs are combined naively without acknowledging design differences

# run jags
jagsfit2 <- crossnma(mod2,
  n.adapt = 500, n.iter = 5000, n.burnin = 2000,
  thin = 1, n.chains = 2)

knitr::kable(summary(jagsfit2, exp = FALSE), digits = 3)

league(jagsfit2, exp = TRUE, cov1.value = 38)
#>                                                               
#>                     A 0.51 (0.18 to 1.24)  0.86 (0.31 to 2.17)
#>   1.97 (0.80 to 5.66)                   B  1.69 (0.57 to 5.36)
#>   1.16 (0.46 to 3.22) 0.59 (0.19 to 1.75)                    C
#>  2.62 (0.75 to 11.22) 1.33 (0.32 to 6.22) 2.23 (0.52 to 10.72)
#>                     
#>  0.38 (0.09 to 1.33)
#>  0.75 (0.16 to 3.17)
#>  0.45 (0.09 to 1.94)
#>                    D

league(jagsfit2, exp = TRUE, cov1.value = 38, direction = "long")
#>    Treatment Comparator    median        lci       uci
#> 1          A          A 1.0000000 1.00000000  1.000000
#> 2          B          A 0.5066849 0.17660669  1.244079
#> 3          C          A 0.8609526 0.31079131  2.173197
#> 4          D          A 0.3816317 0.08913192  1.328199
#> 5          A          B 1.9736131 0.80380774  5.662301
#> 6          B          B 1.0000000 1.00000000  1.000000
#> 7          C          B 1.6873405 0.57081189  5.363864
#> 8          D          B 0.7503932 0.16087147  3.165229
#> 9          A          C 1.1615041 0.46015165  3.217593
#> 10         B          C 0.5926486 0.18643279  1.751891
#> 11         C          C 1.0000000 1.00000000  1.000000
#> 12         D          C 0.4477122 0.09330898  1.935859
#> 13         A          D 2.6203274 0.75289946 11.219325
#> 14         B          D 1.3326347 0.31593289  6.216143
#> 15         C          D 2.2335777 0.51656668 10.717083
#> 16         D          D 1.0000000 1.00000000  1.000000

# jags model: code+data
mod3 <- crossnma.model(treat, id, relapse, n, design,
  prt.data = ipddata, std.data = stddata,
  reference = "D", trt.effect = "random",
  #----------  meta-regression ----------
  cov1 = age,
  split.regcoef = FALSE,
  #---------- bias adjustment ----------
  method.bias = "prior",
  run.nrs =
    list(trt.effect = "common",
         var.infl = 0.6, mean.shift = 0,
         n.adapt = 500, n.iter = 10000, n.burnin = 4000,
         thin = 1, n.chains = 2)
  )
#> The data is analyzed assuming the studies have the same design

# run jags
jagsfit3 <- crossnma(mod3,
  n.adapt = 500, n.iter = 5000, n.burnin = 2000,
  thin = 1, n.chains = 2)

heatplot(jagsfit3, exp = TRUE, cov1.value = 38,
  size = 6, size.trt = 20, size.axis = 12)

# jags model: code+data
mod4 <- crossnma.model(treat, id, relapse, n, design,
  prt.data = ipddata, std.data = stddata,
  reference = "A", trt.effect = "random",
  #---------- bias adjustment ----------
  method.bias = 'adjust1',
  bias.type = 'add',
  bias.effect = 'common',
  bias = rob,
  unfav = unfavored,
  bias.group = bias.group,
  bias.covariate = year
)
#> Bias effect is assumed common across studies

# run jags
jagsfit4 <- crossnma(mod4,
  n.adapt = 500, n.iter = 5000, n.burnin = 2000,
  thin = 1, n.chains = 2)

knitr::kable(summary(jagsfit4, exp = FALSE), digits = 3)

# jags model: code+data
mod5 <- crossnma.model(treat, id, relapse, n, design,
  prt.data = ipddata, std.data = stddata,
  reference = "A", trt.effect = "random",
  #---------- bias adjustment ----------
  method.bias = 'adjust2',
  bias.type = 'add',
  bias = rob,
  unfav = unfavored,
  bias.group = bias.group,
)
#> Bias effect is assumed common across studies

# run jags
jagsfit5 <- crossnma(mod5,
  n.adapt = 500, n.iter = 5000, n.burnin = 2000,
  thin = 1, n.chains = 2)

knitr::kable(summary(jagsfit5, exp = FALSE), digits = 3)

par(oldpar)