Adaptive Subgroup Selection in Sequential Trials

The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

Introduction

ASSISTant is an R package for Adaptive Subgroup Selection In Sequential Trials. This vignette reproduces all the simulations in the original paper of Lai, Lavori and Liao (2014).

NOTE The number of simulations has been drastically reduced in this vignette in order to avoid taxing CRAN servers. The full_doc sources contain the complete sources and output; see files in the directory

system.file("full_doc", package="ASSISTant")

library(ASSISTant)
data(LLL.SETTINGS)
str(LLL.SETTINGS)

## List of 3
##  $ trialParameters:List of 4
##   ..$ N         : num [1:3] 300 400 500
##   ..$ type1Error: num 0.05
##   ..$ eps       : num 0.5
##   ..$ type2Error: num 0.2
##  $ scenarios      :List of 11
##   ..$ S0 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S1 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0.3 0 0.3 0 0.3 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S2 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0.3 0 0.3 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S3 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S4 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0.3 0 0.3 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S5 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.6 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S6 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S7 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S8 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 2 1 1.5 1 1 1 0.5 1 0.5 ...
##   ..$ S9 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 0.5 1 1 1 1.5 1 2 1 2 ...
##   ..$ S10:List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0 0 0.3 0 0.4 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##  $ prevalences    :List of 2
##   ..$ table1: num [1:6] 0.167 0.167 0.167 0.167 0.167 ...
##   ..$ table2: num [1:6] 0.2 0.1 0.3 0.1 0.1 0.2

The LLL.SETTINGS list contains all the scenarios described in the paper.

Table 1 Results

Scenario S0

This is the null setting.

scenario <- LLL.SETTINGS$scenarios$S0
designParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
designA <- ASSISTDesign$new(trialParameters = LLL.SETTINGS$trialParameters,
                            designParameters = designParameters)
print(designA)

## Design Parameters:
##  Number of Groups: 6
##  Prevalence:
## 
## |    Group1|    Group2|    Group3|    Group4|    Group5|    Group6|
## |---------:|---------:|---------:|---------:|---------:|---------:|
## | 0.1666667| 0.1666667| 0.1666667| 0.1666667| 0.1666667| 0.1666667|
## 
##  Using Discrete Rankin scores? FALSE
## 
##  Normal Rankin Distribution means (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      0|      0|      0|      0|      0|      0|
## |Alt  |      0|      0|      0|      0|      0|      0|
## 
##  Normal Rankin Distribution SDs (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      1|      1|      1|      1|      1|      1|
## |Alt  |      1|      1|      1|      1|      1|      1|
## 
## Trial Parameters:
## List of 5
##  $ N         : num [1:3] 300 400 500
##  $ type1Error: num 0.05
##  $ eps       : num 0.5
##  $ type2Error: num 0.2
##  $ effectSize: num 0.0642
## 
## Boundaries:
## 
## 
## |    btilde|        b|        c|
## |---------:|--------:|--------:|
## | -1.460993| 2.390404| 2.491775|

result <- designA$explore(numberOfSimulations = 50, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.020000; P(Reject H0_subgp) = 0.000000; P(Reject H0) = 0.020000
## P(Early stop for efficacy [futility]) = 0.020000 [0.640000]
## Mean [SD] Randomized N = 414.000000 [72.871147]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.20|
## |         2|       0.46|
## |         3|       0.34|
## 
## Mean [SD] Lost N = 183.880000 [85.293644]
## Mean [SD] Analyzed N = 230.120000 [92.989716]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 250.94118|  4.955686|
## |             1|             2| 201.42857|  6.241184|
## |             1|             3| 152.33333|  3.511885|
## |             1|             4| 107.66667|  6.027714|
## |             1|             5|  52.80000|  4.266146|
## |             2|             1| 328.00000| 12.727922|
## |             2|             2| 267.33333| 25.026653|
## |             2|             5|  71.66667| 15.143756|
## |             3|             3| 232.00000|        NA|
## |             3|             4| 161.50000|  9.192388|
## |             3|             5|  82.00000| 12.529964|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     6|     1|       0.02|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 17|  7|  3|  3|  5|
## |             2|  2|  3|  0|  0|  3|
## |             3|  0|  0|  1|  2|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.98|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |        1|            19|             0|
## |        1|            10|             0|
## |        1|             4|             0|
## |        1|             5|             0|
## |        1|            11|             0|
## |        0|             1|             1|
## NULL

Alternative Scenario S1

scenario <- LLL.SETTINGS$scenarios$S1
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.780000; P(Reject H0_subgp) = 0.040000; P(Reject H0) = 0.820000
## P(Early stop for efficacy [futility]) = 0.700000 [0.000000]
## Mean [SD] Randomized N = 372.000000 [90.441096]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.58|
## |         2|       0.12|
## |         3|       0.30|
## 
## Mean [SD] Lost N = 45.500000 [112.449762]
## Mean [SD] Analyzed N = 326.500000 [94.545368]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|       sd|
## |-------------:|-------------:|--------:|--------:|
## |             1|             5|  48.0000|       NA|
## |             2|             1| 330.0000|       NA|
## |             3|             1| 415.6667| 8.504900|
## |             3|             3| 246.0000|       NA|
## |             3|             5|  80.8000| 8.228001|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     3|     1|       0.02|
## |     6|    39|       0.78|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  0|  0|  0|  1|
## |             2|  1|  0|  0|  0|  0|
## |             3|  3|  0|  1|  0|  5|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.18|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |     0.75|             4|             1|
## |     0.00|             1|             1|
## |     1.00|             6|             0|
## |     0.00|            39|            39|
## NULL

Alternative Scenario S2

scenario <- LLL.SETTINGS$scenarios$S2
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.400000; P(Reject H0_subgp) = 0.320000; P(Reject H0) = 0.720000
## P(Early stop for efficacy [futility]) = 0.420000 [0.000000]
## Mean [SD] Randomized N = 436.000000 [82.709228]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.22|
## |         2|       0.20|
## |         3|       0.58|
## 
## Mean [SD] Lost N = 122.160000 [123.479381]
## Mean [SD] Analyzed N = 313.840000 [100.538014]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 254.0000|  2.828427|
## |             1|             2| 199.0000|  8.485281|
## |             1|             3| 141.5000| 21.920310|
## |             1|             4|  96.0000| 11.313709|
## |             1|             5|  50.0000|        NA|
## |             2|             4| 133.0000|  5.656854|
## |             3|             1| 423.5000|  7.778175|
## |             3|             2| 334.3333|  7.505554|
## |             3|             3| 241.2500|  6.701990|
## |             3|             4| 167.6667|  5.267827|
## |             3|             5|  87.0000|        NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     2|       0.04|
## |     2|     4|       0.08|
## |     3|     4|       0.08|
## |     4|     6|       0.12|
## |     6|    20|       0.40|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  2|  2|  2|  2|  1|
## |             2|  0|  0|  0|  2|  0|
## |             3|  2|  3|  4|  9|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.28|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.5000000|             4|             2|
## | 0.2000000|             5|             4|
## | 0.3333333|             6|             4|
## | 0.5384615|            13|             6|
## | 1.0000000|             2|             0|
## | 0.0000000|            20|            20|
## NULL

Alternative Scenario S3

scenario <- LLL.SETTINGS$scenarios$S3
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.120000; P(Reject H0_subgp) = 0.420000; P(Reject H0) = 0.540000
## P(Early stop for efficacy [futility]) = 0.260000 [0.020000]
## Mean [SD] Randomized N = 460.000000 [69.985421]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.16|
## |         3|       0.72|
## 
## Mean [SD] Lost N = 223.540000 [126.875837]
## Mean [SD] Analyzed N = 236.460000 [122.725546]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|         sd|
## |-------------:|-------------:|--------:|----------:|
## |             1|             1| 251.6667|  4.3204938|
## |             1|             2| 199.7500|  7.8898669|
## |             1|             3| 142.0000|  8.4852814|
## |             1|             4| 108.0000|         NA|
## |             1|             5|  49.0000|         NA|
## |             2|             1| 335.7500|  6.7515430|
## |             2|             2| 267.0000|         NA|
## |             2|             3| 209.0000|         NA|
## |             2|             5|  65.0000|         NA|
## |             3|             1| 416.8000|  5.8480766|
## |             3|             2| 334.3750|  9.8841504|
## |             3|             3| 261.6667| 11.5902258|
## |             3|             4| 166.8000|  9.8843310|
## |             3|             5|  82.5000|  0.7071068|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    12|       0.24|
## |     2|     6|       0.12|
## |     3|     2|       0.04|
## |     5|     1|       0.02|
## |     6|     6|       0.12|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  6|  4|  2|  1|  1|
## |             2|  4|  1|  1|  0|  1|
## |             3|  5|  8|  3|  5|  2|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.46|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.2000000|            15|            12|
## | 0.5384615|            13|             6|
## | 0.6666667|             6|             2|
## | 1.0000000|             6|             0|
## | 0.7500000|             4|             1|
## | 0.0000000|             6|             6|
## NULL

Alternative Scenario S4

scenario <- LLL.SETTINGS$scenarios$S4
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.060000; P(Reject H0_subgp) = 0.100000; P(Reject H0) = 0.160000
## P(Early stop for efficacy [futility]) = 0.040000 [0.180000]
## Mean [SD] Randomized N = 466.000000 [68.838765]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.10|
## |         3|       0.78|
## 
## Mean [SD] Lost N = 146.480000 [84.999457]
## Mean [SD] Analyzed N = 319.520000 [101.819015]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 251.80000|  1.483240|
## |             1|             4| 100.25000|  8.811518|
## |             1|             5|  52.66667|  1.527525|
## |             2|             4| 130.00000|  8.366600|
## |             2|             5|  66.50000| 24.748737|
## |             3|             1| 416.00000|        NA|
## |             3|             3| 251.66667|  8.524475|
## |             3|             4| 170.91667| 11.712141|
## |             3|             5|  79.16667|  5.980524|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     3|     1|       0.02|
## |     4|     1|       0.02|
## |     5|     2|       0.04|
## |     6|     3|       0.06|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  5|  0|  0|  8|  3|
## |             2|  0|  0|  0|  4|  2|
## |             3|  1|  0|  6| 12|  6|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.84|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.8333333|             6|             1|
## | 0.8333333|             6|             1|
## | 0.9583333|            24|             1|
## | 0.8181818|            11|             2|
## | 0.0000000|             3|             3|
## NULL

Alternative Scenario S5

scenario <- LLL.SETTINGS$scenarios$S5
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.140000; P(Reject H0_subgp) = 0.660000; P(Reject H0) = 0.800000
## P(Early stop for efficacy [futility]) = 0.460000 [0.000000]
## Mean [SD] Randomized N = 444.000000 [67.491496]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.10|
## |         2|       0.36|
## |         3|       0.54|
## 
## Mean [SD] Lost N = 270.120000 [141.724314]
## Mean [SD] Analyzed N = 173.880000 [126.469427]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 248.6923|  5.807820|
## |             1|             2| 206.5000|  2.121320|
## |             1|             3| 157.0000|  4.242641|
## |             1|             4| 104.5000|  4.949747|
## |             2|             1| 333.3333|  4.457204|
## |             3|             1| 417.3125|  5.313113|
## |             3|             2| 330.0000| 12.727922|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    30|       0.60|
## |     2|     1|       0.02|
## |     3|     1|       0.02|
## |     4|     1|       0.02|
## |     6|     7|       0.14|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 13|  2|  2|  2|  0|
## |             2|  6|  0|  0|  0|  0|
## |             3| 16|  2|  0|  0|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |     0.2|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1428571|            35|            30|
## | 0.7500000|             4|             1|
## | 0.5000000|             2|             1|
## | 0.5000000|             2|             1|
## | 0.0000000|             7|             7|
## NULL

Alternative Scenario S6

scenario <- LLL.SETTINGS$scenarios$S6
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.060000; P(Reject H0_subgp) = 0.720000; P(Reject H0) = 0.780000
## P(Early stop for efficacy [futility]) = 0.480000 [0.040000]
## Mean [SD] Randomized N = 436.000000 [69.282032]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.40|
## |         3|       0.48|
## 
## Mean [SD] Lost N = 254.060000 [123.124196]
## Mean [SD] Analyzed N = 181.940000 [115.743808]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 249.85714|  7.336496|
## |             1|             2| 189.66667|  5.507570|
## |             1|             4| 102.66667| 11.718931|
## |             1|             5|  49.00000|  8.485281|
## |             2|             1| 330.42857|  6.106203|
## |             2|             2| 267.00000| 14.142136|
## |             3|             1| 416.55556|  3.574602|
## |             3|             2| 336.75000| 12.685293|
## |             3|             5|  95.66667|  7.023769|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    26|       0.52|
## |     2|     6|       0.12|
## |     4|     1|       0.02|
## |     5|     3|       0.06|
## |     6|     3|       0.06|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 14|  3|  0|  3|  2|
## |             2|  7|  2|  0|  0|  0|
## |             3|  9|  4|  0|  0|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.22|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1333333|            30|            26|
## | 0.3333333|             9|             6|
## | 0.6666667|             3|             1|
## | 0.4000000|             5|             3|
## | 0.0000000|             3|             3|
## NULL

Alternative Scenario S7

scenario <- LLL.SETTINGS$scenarios$S7
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.420000; P(Reject H0_subgp) = 0.480000; P(Reject H0) = 0.900000
## P(Early stop for efficacy [futility]) = 0.460000 [0.000000]
## Mean [SD] Randomized N = 428.000000 [85.809471]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.20|
## |         3|       0.54|
## 
## Mean [SD] Lost N = 137.180000 [132.051735]
## Mean [SD] Analyzed N = 290.820000 [102.494338]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|         sd|
## |-------------:|-------------:|--------:|----------:|
## |             1|             1| 243.5000| 12.0208153|
## |             1|             2| 198.7500|  5.5000000|
## |             1|             3| 153.5000|  0.7071068|
## |             1|             4|  95.0000|         NA|
## |             2|             3| 206.0000|  8.4852814|
## |             3|             1| 422.0000|         NA|
## |             3|             2| 341.0000|  6.1644140|
## |             3|             3| 257.0000|  5.0709255|
## |             3|             4| 165.6667|  1.5275252|
## |             3|             5|  83.0000|         NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     3|       0.06|
## |     2|     9|       0.18|
## |     3|     9|       0.18|
## |     4|     3|       0.06|
## |     6|    21|       0.42|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  2|  4|  2|  1|  0|
## |             2|  0|  0|  2|  0|  0|
## |             3|  1|  5|  8|  3|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |     0.1|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |     0.00|             3|             3|
## |     0.00|             9|             9|
## |     0.25|            12|             9|
## |     0.25|             4|             3|
## |     1.00|             1|             0|
## |     0.00|            21|            21|
## NULL

Alternative Scenario S8

scenario <- LLL.SETTINGS$scenarios$S8
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.180000; P(Reject H0_subgp) = 0.500000; P(Reject H0) = 0.680000
## P(Early stop for efficacy [futility]) = 0.260000 [0.000000]
## Mean [SD] Randomized N = 460.000000 [72.843136]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.14|
## |         2|       0.12|
## |         3|       0.74|
## 
## Mean [SD] Lost N = 170.720000 [104.977556]
## Mean [SD] Analyzed N = 289.280000 [85.082682]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 250.0000|  5.291503|
## |             1|             3| 147.6000|  2.509980|
## |             1|             4| 106.0000|  6.055301|
## |             2|             1| 322.0000|        NA|
## |             2|             3| 207.0000|        NA|
## |             3|             2| 331.3333| 11.590226|
## |             3|             3| 250.7647| 10.195155|
## |             3|             4| 160.0000|  1.732051|
## |             3|             5|  89.5000|  3.109126|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     4|       0.08|
## |     2|     3|       0.06|
## |     3|    14|       0.28|
## |     4|     3|       0.06|
## |     5|     1|       0.02|
## |     6|     9|       0.18|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  3|  0|  5|  4|  0|
## |             2|  1|  0|  1|  0|  0|
## |             3|  0|  3| 17|  3|  4|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.32|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             4|             4|
## | 0.0000000|             3|             3|
## | 0.3913043|            23|            14|
## | 0.5714286|             7|             3|
## | 0.7500000|             4|             1|
## | 0.0000000|             9|             9|
## NULL

Alternative Scenario S9

scenario <- LLL.SETTINGS$scenarios$S9
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.560000; P(Reject H0_subgp) = 0.360000; P(Reject H0) = 0.920000
## P(Early stop for efficacy [futility]) = 0.480000 [0.000000]
## Mean [SD] Randomized N = 426.000000 [85.260705]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.22|
## |         3|       0.52|
## 
## Mean [SD] Lost N = 125.840000 [153.421968]
## Mean [SD] Analyzed N = 300.160000 [127.468341]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 235.0000|        NA|
## |             1|             2| 194.0000|        NA|
## |             2|             1| 329.6667|  2.081666|
## |             3|             1| 418.5000|  4.949747|
## |             3|             2| 330.6250| 13.394429|
## |             3|             3| 243.2500|  7.365460|
## |             3|             4| 168.0000|  1.414214|
## |             3|             5|  83.0000|        NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     5|       0.10|
## |     2|     8|       0.16|
## |     3|     3|       0.06|
## |     4|     1|       0.02|
## |     5|     1|       0.02|
## |     6|    28|       0.56|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  1|  0|  0|  0|
## |             2|  3|  0|  0|  0|  0|
## |             3|  2|  8|  4|  2|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.08|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1666667|             6|             5|
## | 0.1111111|             9|             8|
## | 0.2500000|             4|             3|
## | 0.5000000|             2|             1|
## | 0.0000000|             1|             1|
## | 0.0000000|            28|            28|
## NULL

Alternative Scenario S10

scenario <- LLL.SETTINGS$scenarios$S10
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.420000; P(Reject H0_subgp) = 0.000000; P(Reject H0) = 0.420000
## P(Early stop for efficacy [futility]) = 0.300000 [0.160000]
## Mean [SD] Randomized N = 422.000000 [91.003476]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.32|
## |         2|       0.14|
## |         3|       0.54|
## 
## Mean [SD] Lost N = 78.100000 [118.835206]
## Mean [SD] Analyzed N = 343.900000 [117.819345]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|       sd|
## |-------------:|-------------:|---------:|--------:|
## |             1|             1| 252.00000|       NA|
## |             1|             5|  50.42857| 4.035556|
## |             3|             1| 418.25000| 9.979145|
## |             3|             2| 331.00000|       NA|
## |             3|             5|  81.00000| 8.074652|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     6|    21|       0.42|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  0|  0|  0|  7|
## |             2|  0|  0|  0|  0|  0|
## |             3|  4|  1|  0|  0| 16|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.58|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |        1|             5|             0|
## |        1|             1|             0|
## |        1|            23|             0|
## |        0|            21|            21|
## NULL

Table 2 Results

Scenario S0

scenario <- LLL.SETTINGS$scenarios$S0
designParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
designA <- ASSISTDesign$new(trialParameters = LLL.SETTINGS$trialParameters,
                            designParameters = designParameters)
print(designA)

## Design Parameters:
##  Number of Groups: 6
##  Prevalence:
## 
## | Group1| Group2| Group3| Group4| Group5| Group6|
## |------:|------:|------:|------:|------:|------:|
## |    0.2|    0.1|    0.3|    0.1|    0.1|    0.2|
## 
##  Using Discrete Rankin scores? FALSE
## 
##  Normal Rankin Distribution means (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      0|      0|      0|      0|      0|      0|
## |Alt  |      0|      0|      0|      0|      0|      0|
## 
##  Normal Rankin Distribution SDs (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      1|      1|      1|      1|      1|      1|
## |Alt  |      1|      1|      1|      1|      1|      1|
## 
## Trial Parameters:
## List of 5
##  $ N         : num [1:3] 300 400 500
##  $ type1Error: num 0.05
##  $ eps       : num 0.5
##  $ type2Error: num 0.2
##  $ effectSize: num 0.0642
## 
## Boundaries:
## 
## 
## |    btilde|        b|        c|
## |---------:|--------:|--------:|
## | -1.460993| 2.371573| 2.467677|

result <- designA$explore(numberOfSimulations = 50, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.020000; P(Reject H0_subgp) = 0.020000; P(Reject H0) = 0.040000
## P(Early stop for efficacy [futility]) = 0.040000 [0.620000]
## Mean [SD] Randomized N = 406.000000 [79.308234]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.28|
## |         2|       0.38|
## |         3|       0.34|
## 
## Mean [SD] Lost N = 175.400000 [80.752835]
## Mean [SD] Analyzed N = 230.600000 [105.285811]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 240.0000|  9.518403|
## |             1|             2| 211.3333|  7.364946|
## |             1|             3| 116.0000| 16.970563|
## |             1|             4|  86.0000|  7.071068|
## |             1|             5|  59.0000|  3.162278|
## |             2|             1| 325.0000|        NA|
## |             2|             2| 283.0000|  4.242641|
## |             2|             3| 158.5000| 14.849242|
## |             2|             4| 111.0000|        NA|
## |             2|             5|  94.0000|        NA|
## |             3|             1| 369.0000|        NA|
## |             3|             3| 192.0000|  7.071068|
## |             3|             4| 156.0000|        NA|
## |             3|             5| 105.0000|  7.937254|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     2|     1|       0.02|
## |     6|     1|       0.02|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 11| 12|  2|  5|  5|
## |             2|  1|  2|  2|  1|  1|
## |             3|  1|  0|  2|  1|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.96|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 1.0000000|            13|             0|
## | 0.9285714|            14|             1|
## | 1.0000000|             6|             0|
## | 1.0000000|             7|             0|
## | 1.0000000|             9|             0|
## | 0.0000000|             1|             1|
## NULL

Alternative Scenario S1

scenario <- LLL.SETTINGS$scenarios$S1
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.780000; P(Reject H0_subgp) = 0.060000; P(Reject H0) = 0.840000
## P(Early stop for efficacy [futility]) = 0.760000 [0.000000]
## Mean [SD] Randomized N = 366.000000 [84.780629]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.58|
## |         2|       0.18|
## |         3|       0.24|
## 
## Mean [SD] Lost N = 44.200000 [97.398403]
## Mean [SD] Analyzed N = 321.800000 [67.726340]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|       sd|
## |-------------:|-------------:|--------:|--------:|
## |             2|             2| 282.0000|       NA|
## |             3|             2| 347.6667| 4.725816|
## |             3|             3| 205.0000|       NA|
## |             3|             4| 133.5000| 3.535534|
## |             3|             5| 103.2500| 1.707825|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     2|     1|       0.02|
## |     3|     1|       0.02|
## |     5|     1|       0.02|
## |     6|    39|       0.78|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  0|  0|  0|  0|
## |             2|  0|  1|  0|  0|  0|
## |             3|  0|  3|  1|  2|  4|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.16|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |     0.75|             4|             1|
## |     0.00|             1|             1|
## |     1.00|             2|             0|
## |     0.75|             4|             1|
## |     0.00|            39|            39|
## NULL

Alternative Scenario S2

scenario <- LLL.SETTINGS$scenarios$S2
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.520000; P(Reject H0_subgp) = 0.220000; P(Reject H0) = 0.740000
## P(Early stop for efficacy [futility]) = 0.520000 [0.020000]
## Mean [SD] Randomized N = 424.000000 [79.693290]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.22|
## |         2|       0.32|
## |         3|       0.46|
## 
## Mean [SD] Lost N = 79.800000 [98.886866]
## Mean [SD] Analyzed N = 344.200000 [80.416264]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 237.00000|        NA|
## |             1|             2| 222.00000|        NA|
## |             1|             3| 122.00000|        NA|
## |             1|             4|  87.00000|  5.567764|
## |             2|             2| 281.00000|        NA|
## |             2|             4| 137.00000|        NA|
## |             3|             2| 353.00000|  4.242641|
## |             3|             3| 198.00000| 26.870058|
## |             3|             4| 149.00000| 11.302655|
## |             3|             5|  95.66667|  5.686241|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     2|     1|       0.02|
## |     4|     8|       0.16|
## |     5|     1|       0.02|
## |     6|    26|       0.52|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  1|  1|  3|  0|
## |             2|  0|  1|  0|  1|  0|
## |             3|  0|  2|  2|  9|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.26|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             1|             1|
## | 0.7500000|             4|             1|
## | 1.0000000|             3|             0|
## | 0.3846154|            13|             8|
## | 0.6666667|             3|             1|
## | 0.0000000|            26|            26|
## NULL

Alternative Scenario S3

scenario <- LLL.SETTINGS$scenarios$S3
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.060000; P(Reject H0_subgp) = 0.560000; P(Reject H0) = 0.620000
## P(Early stop for efficacy [futility]) = 0.300000 [0.040000]
## Mean [SD] Randomized N = 458.000000 [64.174507]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.08|
## |         2|       0.26|
## |         3|       0.66|
## 
## Mean [SD] Lost N = 244.860000 [104.019035]
## Mean [SD] Analyzed N = 213.140000 [90.550022]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|         sd|
## |-------------:|-------------:|--------:|----------:|
## |             1|             1| 240.1250|  7.7170960|
## |             1|             2| 209.8000|  7.5099933|
## |             1|             4| 100.0000|         NA|
## |             2|             1| 323.0000|  4.2426407|
## |             2|             2| 277.5000|  6.3639610|
## |             2|             4| 116.0000|         NA|
## |             2|             5|  75.0000|         NA|
## |             3|             1| 401.5000|  2.1213203|
## |             3|             2| 345.6667| 12.6923299|
## |             3|             3| 187.5000|  0.7071068|
## |             3|             4| 151.0000|         NA|
## |             3|             5| 109.0000|  1.4142136|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     9|       0.18|
## |     2|    18|       0.36|
## |     4|     1|       0.02|
## |     6|     3|       0.06|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  8| 10|  0|  1|  0|
## |             2|  2|  2|  0|  1|  1|
## |             3|  2| 15|  2|  1|  2|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.38|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.2500000|            12|             9|
## | 0.3333333|            27|            18|
## | 1.0000000|             2|             0|
## | 0.6666667|             3|             1|
## | 1.0000000|             3|             0|
## | 0.0000000|             3|             3|
## NULL

Alternative Scenario S4

scenario <- LLL.SETTINGS$scenarios$S4
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.160000; P(Reject H0_subgp) = 0.120000; P(Reject H0) = 0.280000
## P(Early stop for efficacy [futility]) = 0.140000 [0.160000]
## Mean [SD] Randomized N = 452.000000 [78.869564]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.18|
## |         2|       0.12|
## |         3|       0.70|
## 
## Mean [SD] Lost N = 103.560000 [60.185143]
## Mean [SD] Analyzed N = 348.440000 [65.372140]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 238.00000|        NA|
## |             1|             3| 127.00000|  8.485281|
## |             1|             4|  96.33333|  8.594572|
## |             1|             5|  61.50000|  9.071935|
## |             3|             3| 196.00000| 22.538855|
## |             3|             4| 151.64286|  7.601692|
## |             3|             5| 102.80000| 10.347302|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     4|     5|       0.10|
## |     5|     1|       0.02|
## |     6|     8|       0.16|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  0|  2|  6|  6|
## |             2|  0|  0|  0|  0|  0|
## |             3|  0|  0|  3| 14| 10|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.72|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |   1.0000|             1|             0|
## |   1.0000|             5|             0|
## |   0.7500|            20|             5|
## |   0.9375|            16|             1|
## |   0.0000|             8|             8|
## NULL

Alternative Scenario S5

scenario <- LLL.SETTINGS$scenarios$S5
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.100000; P(Reject H0_subgp) = 0.740000; P(Reject H0) = 0.840000
## P(Early stop for efficacy [futility]) = 0.400000 [0.000000]
## Mean [SD] Randomized N = 446.000000 [73.429127]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.14|
## |         2|       0.26|
## |         3|       0.60|
## 
## Mean [SD] Lost N = 282.580000 [122.120230]
## Mean [SD] Analyzed N = 163.420000 [104.629783]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 238.8182|  7.769404|
## |             1|             2| 200.3333| 10.503967|
## |             2|             1| 330.0000|  9.183318|
## |             2|             2| 278.0000|        NA|
## |             2|             4| 113.0000|        NA|
## |             3|             1| 399.2667| 10.746871|
## |             3|             2| 357.6000|  6.655824|
## |             3|             3| 211.5000|  3.535534|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    28|       0.56|
## |     2|     8|       0.16|
## |     3|     1|       0.02|
## |     6|     5|       0.10|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 11|  3|  0|  0|  0|
## |             2|  7|  1|  0|  1|  0|
## |             3| 15|  5|  2|  0|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.16|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1515152|            33|            28|
## | 0.1111111|             9|             8|
## | 0.5000000|             2|             1|
## | 1.0000000|             1|             0|
## | 0.0000000|             5|             5|
## NULL

Alternative Scenario S6

scenario <- LLL.SETTINGS$scenarios$S6
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.080000; P(Reject H0_subgp) = 0.680000; P(Reject H0) = 0.760000
## P(Early stop for efficacy [futility]) = 0.320000 [0.000000]
## Mean [SD] Randomized N = 456.000000 [70.450446]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.20|
## |         3|       0.68|
## 
## Mean [SD] Lost N = 275.360000 [129.742722]
## Mean [SD] Analyzed N = 180.640000 [114.948215]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 236.8750|  7.453427|
## |             1|             2| 199.5000| 13.435029|
## |             1|             3| 126.0000|        NA|
## |             1|             4| 106.0000|        NA|
## |             1|             5|  61.0000|        NA|
## |             2|             1| 326.8000|  6.648308|
## |             2|             4| 134.0000|        NA|
## |             3|             1| 395.0526|  9.507153|
## |             3|             2| 359.0000| 14.000000|
## |             3|             3| 204.3333|  8.504900|
## |             3|             4| 132.0000|        NA|
## |             3|             5|  85.0000|        NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    26|       0.52|
## |     2|     5|       0.10|
## |     3|     3|       0.06|
## |     6|     4|       0.08|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  8|  2|  1|  1|  1|
## |             2|  5|  0|  0|  1|  0|
## |             3| 19|  3|  3|  1|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.24|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |   0.1875|            32|            26|
## |   0.0000|             5|             5|
## |   0.2500|             4|             3|
## |   1.0000|             3|             0|
## |   1.0000|             2|             0|
## |   0.0000|             4|             4|
## NULL

Alternative Scenario S7

scenario <- LLL.SETTINGS$scenarios$S7
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.580000; P(Reject H0_subgp) = 0.380000; P(Reject H0) = 0.960000
## P(Early stop for efficacy [futility]) = 0.540000 [0.000000]
## Mean [SD] Randomized N = 416.000000 [86.567185]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.30|
## |         2|       0.24|
## |         3|       0.46|
## 
## Mean [SD] Lost N = 104.820000 [132.987446]
## Mean [SD] Analyzed N = 311.180000 [105.744144]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|       sd|
## |-------------:|-------------:|--------:|--------:|
## |             1|             2| 212.5000|  5.00000|
## |             1|             3| 123.0000|       NA|
## |             2|             2| 290.0000|       NA|
## |             3|             1| 393.0000|       NA|
## |             3|             2| 345.2000| 10.94075|
## |             3|             3| 206.5556| 12.74864|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     2|    10|       0.20|
## |     3|     8|       0.16|
## |     6|    29|       0.58|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  4|  1|  0|  0|
## |             2|  0|  1|  0|  0|  0|
## |             3|  1|  5|  9|  0|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.04|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |      0.0|             1|             1|
## |      0.0|            10|            10|
## |      0.2|            10|             8|
## |      0.0|            29|            29|
## NULL

Alternative Scenario S8

scenario <- LLL.SETTINGS$scenarios$S8
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.360000; P(Reject H0_subgp) = 0.480000; P(Reject H0) = 0.840000
## P(Early stop for efficacy [futility]) = 0.400000 [0.000000]
## Mean [SD] Randomized N = 438.000000 [83.029378]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.22|
## |         2|       0.18|
## |         3|       0.60|
## 
## Mean [SD] Lost N = 121.880000 [114.118563]
## Mean [SD] Analyzed N = 316.120000 [89.724440]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|  mean|        sd|
## |-------------:|-------------:|-----:|---------:|
## |             1|             2| 209.0| 12.727922|
## |             1|             3| 120.0|  5.385165|
## |             1|             4|  87.0|        NA|
## |             3|             1| 393.0|        NA|
## |             3|             2| 350.5| 11.030261|
## |             3|             3| 198.0| 11.162437|
## |             3|             4| 153.5|  4.203173|
## |             3|             5| 100.5|  9.110434|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     2|     5|       0.10|
## |     3|    11|       0.22|
## |     4|     3|       0.06|
## |     5|     4|       0.08|
## |     6|    18|       0.36|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  2|  5|  1|  0|
## |             2|  0|  0|  0|  0|  0|
## |             3|  1|  4| 11|  4|  4|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.16|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             1|             1|
## | 0.1666667|             6|             5|
## | 0.3125000|            16|            11|
## | 0.4000000|             5|             3|
## | 0.0000000|             4|             4|
## | 0.0000000|            18|            18|
## NULL

Alternative Scenario S9

scenario <- LLL.SETTINGS$scenarios$S9
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.540000; P(Reject H0_subgp) = 0.420000; P(Reject H0) = 0.960000
## P(Early stop for efficacy [futility]) = 0.480000 [0.000000]
## Mean [SD] Randomized N = 426.000000 [85.260705]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.22|
## |         3|       0.52|
## 
## Mean [SD] Lost N = 124.200000 [148.980207]
## Mean [SD] Analyzed N = 301.800000 [115.994370]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 234.0000|        NA|
## |             1|             2| 219.5000|  2.121320|
## |             1|             5|  65.0000|        NA|
## |             2|             2| 288.0000|        NA|
## |             3|             1| 384.0000|  2.828427|
## |             3|             2| 346.6667| 15.491933|
## |             3|             3| 195.4000|  7.300685|
## |             3|             4| 159.5000| 10.606602|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     3|       0.06|
## |     2|    11|       0.22|
## |     3|     5|       0.10|
## |     4|     2|       0.04|
## |     6|    27|       0.54|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  2|  0|  0|  1|
## |             2|  0|  1|  0|  0|  0|
## |             3|  2|  9|  5|  2|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.04|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             3|             3|
## | 0.0833333|            12|            11|
## | 0.0000000|             5|             5|
## | 0.0000000|             2|             2|
## | 1.0000000|             1|             0|
## | 0.0000000|            27|            27|
## NULL

Alternative Scenario S10

scenario <- LLL.SETTINGS$scenarios$S10
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.280000; P(Reject H0_subgp) = 0.020000; P(Reject H0) = 0.300000
## P(Early stop for efficacy [futility]) = 0.260000 [0.280000]
## Mean [SD] Randomized N = 420.000000 [83.299313]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.28|
## |         3|       0.46|
## 
## Mean [SD] Lost N = 105.280000 [101.708876]
## Mean [SD] Analyzed N = 314.720000 [98.752430]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 237.0000|  5.244044|
## |             1|             2| 215.0000|  8.869423|
## |             1|             5|  56.5000|  7.148426|
## |             2|             1| 325.0000|        NA|
## |             2|             5|  84.0000|        NA|
## |             3|             1| 400.0000|        NA|
## |             3|             2| 359.0000|        NA|
## |             3|             5| 100.7059| 10.042190|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     5|     1|       0.02|
## |     6|    14|       0.28|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  5|  4|  0|  0|  6|
## |             2|  1|  0|  0|  0|  1|
## |             3|  1|  1|  0|  0| 17|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |     0.7|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 1.0000000|             7|             0|
## | 1.0000000|             5|             0|
## | 0.9583333|            24|             1|
## | 0.0000000|            14|            14|
## NULL

References

Lai, Tze Leung, Philip W. Lavori, and Olivia Yueh-Wen Liao. 2014. “Adaptive Choice of Patient Subgroup for Comparing Two Treatments.” Contemporary Clinical Trials 39 (2): 191–200. https://doi.org/10.1016/j.cct.2014.09.001.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.

Adaptive Subgroup Selection in Sequential Trials

Tze Leung Lai, Philip W. Lavori, Olivia Liao, Ka Wai Tsang and Balasubramanian Narasimhan

2022-12-01

Introduction

Table 1 Results

Scenario S0

Alternative Scenario S1

Alternative Scenario S2

Alternative Scenario S3

Alternative Scenario S4

Alternative Scenario S5

Alternative Scenario S6

Alternative Scenario S7

Alternative Scenario S8

Alternative Scenario S9

Alternative Scenario S10

Table 2 Results

Scenario S0

Alternative Scenario S1

Alternative Scenario S2

Alternative Scenario S3

Alternative Scenario S4

Alternative Scenario S5

Alternative Scenario S6

Alternative Scenario S7

Alternative Scenario S8

Alternative Scenario S9

Alternative Scenario S10

References