Specify Actions to Execute at Trial Milestones

Perform statistical analyses on data snapshot

Statisticians rarely stop at just locking the data–we want to analyze it!

A milestone provide an opportunity to perform formal analysis on the accumulated data, guiding decisions such as continuing, adapting, or stopping the trial.

Let’s revisit the interim milestone. The following example shows how one might compute hazard ratios for two active arms (low and high dose) compared against placebo. The action function extracts the snapshot of patient-level data, fits separate Cox proportional hazards models, and then computes the hazard ratios.

action_at_interim <- function(trial, milestone_name){
  
  locked_data <- trial$get_locked_data(milestone_name)
  
  ## write your own codes to fit coxph model on locked_data
  fit_high <- coxph(Surv(PFS, PFS_event) ~ arm, 
                    data = locked_data %>% 
                      filter(arm %in% c('pbo', 'high'))
                    )
  
  fit_low <- coxph(Surv(PFS, PFS_event) ~ arm, 
                   data = locked_data %>% 
                     filter(arm %in% c('pbo', 'low'))
                   )
  
  hr <- data.frame(high = exp(coef(fit_high)), 
                   low = exp(coef(fit_low)))
  
  ## write more codes to compute one- or two-sided p-values, etc. 
  
  invisible(NULL)
  
}

This function illustrates the flexibility of action functions: any valid R code can be executed at the milestone. However, writing such functions from scratch may become cumbersome and error-prone, especially when:

• multiple treatment arms exist,

• one-sided tests are required, or

• different trial scenarios have varying arm names or endpoints

To reduce coding burden and promote standardization, TrialSimulator provides a collection of helper functions for common statistical analyses. These wrappers ensure consistent inputs and outputs, making downstream use (such as saving or combining results) much easier.

Here is a rewritten version of the interim action function using these helper functions:

action_at_interim <- function(trial, milestone_name){
  
  locked_data <- trial$get_locked_data(milestone_name)
  
  fit_PFS <- fitCoxph(Surv(PFS, PFS_event) ~ arm, 
                      placebo = 'pbo', 
                      data = locked_data, 
                      alternative = 'less', 
                      scale = 'hazard ratio') # also also request other scales
  
  fit_OS <- fitLogrank(Surv(OS, OS_event) ~ arm, 
                       placebo = 'pbo', 
                       data = locked_data, 
                       alternative = 'less', 
                       biomarker1 == 'positive' & 
                         biomarker2 == 'negative') # specify subgroup through ...
  
  invisible(NULL)
  
}

In this example,

• fitCoxph estimates hazard ratio for PFS against placebo, with a one-sided test.

• figLogrank performs logrank test for OS, restricted to a biomarker-defined subgroup through the argument ... in helper functions.

Both helper functions return results in a standardized format, which integrates seamlessly with other utilities of TrialSimulator. This is particularly useful when saving outputs across milestones or performing multiplicity adjustments.

For more complete description of these helper functions for common statistical analyses, see the vignette Wrapper Functions of Common Statistical Methods in TrialSimulator.

At this stage, we are only calculating statistics. In the next section, we will show how to save these results for later use—either to summarize operating characteristics or to carry information forward into future milestones.

Save intermediate results

TrialSimulator offers dedicated member functions for saving intermediate results during a trial. The stored information can be broadly classified into two categories:

• Trial output—results that directly contribute to the summary of trial operating characteristics (e.g., effect estimates, test decisions) or anything that we need for each simulated trial. They are stored in a private single-row data frame within the trial object. We cannot access or modify it directly. Instead, they should be inserted and accessed using member functions trial$save() and trial$get_output(). Since outputs are stored in a single-row data frame, only scalars or single-row data frame are accepted by trial$save()

• auxiliary information—results that may be useful in subsequent analysis (e.g., in action functions of other milestones). Any results that are not directly part of trial-level summaries should be considered as auxiliary information. These are stored using member function trial$save_custom_data(). To retrieve it later, simply call trial$get(). This mechanism provides a safe way to pass data across milestones without relying on global variables. Unlike trial$save(), trial$save_custom_data() can save any objects, which provides flexibility.

In addition, a convenience function trial$bind() is provided for sequentially appending rows to a stored data frame—useful when accumulating results across milestones.

Here we give more details of those member functions. We can also refer to their manuals by running ?Trials in R console.

• save(value, name): saves a scalar or a single-row data frame into the trial’s output.

  • later retrieved with trial$get_output(cols = name).

  • calling trial$get_output() without argument returns all saved outputs available so far.

• save_custom_data(value, name): saves an object of any type as auxiliary information as a temporary and private list within the trial.

  • retrieved later by calling trial$get(name).

• bind(value, name): a special case of save_custom_data for data frames.

  • If the named data frame already exists, the new value is row-bound to it. Otherwise, a new data frame is created and saved.

  • retrieved later by trial$get_custom_data(name).

  • particularly useful for accumulating results across multiple milestones in group sequential designs. For example, we may want to save estimates, p-values, etc. across milestones to a data frame, so that we can test it with a graphical testing strategy when all p-values are collected (i.e. at the last milestone).

action_at_interim <- function(trial, milestone_name){
  
  locked_data <- trial$get_locked_data(milestone_name)
  
  ## fit coxph model on locked_data
  ## extract estimate and p-value of hazard ratio
  ## assume that hr is estimate, and pval is p-value
  trial$save(value = hr, name = 'pfs_interim_hazard_ratio')
  trial$save(value = pval, name = 'pfs_interim_p_value')
  
  ## values of hr and pval can be accessed anywhere later by calling
  ## trial$get_output(cols = 'pfs_interim_hazard_ratio')
  ## trial$get_output(cols = 'pfs_interim_p_value')
  ## trial$get_output(cols = c('pfs_interim_hazard_ratio', 'pfs_interim_hazard_ratio'))
  ## If a colum specified in cols does not exist (e.g., typo in codes), 
  ## an informative error message will be prompted. 
  
  invisible(NULL)
  
}

action_at_interim <- function(trial, milestone_name){
  
  locked_data <- trial$get_locked_data(milestone_name)
  
  ## fit coxph model on locked_data
  ## extract estimate and p-value of hazard ratio
  ## assume that hr is estimate, and pval is p-value
  trial$save(value = hr, name = 'pfs_interim_hazard_ratio')
  trial$save(value = pval, name = 'pfs_interim_p_value')
  
  ## assume that we also compute other values that may be useful in later milestone, 
  ## we save them in a list and call
  more_results <- list(numeric = 1, 
                       string = 'abc', 
                       list = list(e1 = 100, e2 = 'AA'))
  
  trial$save_custom_data(value = more_results, name = 'more_results')
  ## value of more_results can be accessed anywhere later by calling
  ## tmp <- trial$get(name = 'more_results')
  
  ## no return value is needed in an action function
  
}

action_at_final <- function(trial, milestone_name){
  
  locked_data <- trial$get_locked_data(milestone_name)
  
  ## extract outputs
  pfs_hr_interim <- trial$get_output(cols = 'pfs_interim_hazard_ratio')
  pfs_hr_pval <- trial$get_output(cols = 'pfs_interim_p_value')
  
  ## extract auxiliary information
  custom_list <- trial$get('more_results')
  
  invisible(NULL)
  
}

trial <- trial(...) ## initialize a trial with necessary arguments in ...
trial$add_arms(sample_ratio = c(1, 2, 1), pbo, low, high)

trial$save_custom_data(value = 0.025, name = 'fwer')

action_at_interim <- function(trial, milestone_name){
  
  locked_data <- trial$get_locked_data(milestone_name)
  
  ## compute p-values of PFS and OS at interim
  ## then extract FWER
  alpha <- trial$get(name = 'fwer')
  
  ## compute decision boundaries at interim based on FWER
  ## then test PFS and OS
  
  ## save testing results for interim
  # trial$save(...)
  
  invisible(NULL)
  
}

action_at_final <- function(trial, milestone_name){
  
  locked_data <- trial$get_locked_data(milestone_name)
  
  ## compute p-values of PFS and OS at final
  ## then extract FWER and testing result at interim to adjust boundaries
  alpha <- trial$get(name = 'fwer')
  # interim_results <- trial$get_output(...)
  
  ## test PFS and OS again
  
  ## save testing results for final
  # trial$save(...)
  
  invisible(NULL)
  
}

Alter a trial via adaptation

Beyond analyzing data and saving results, one of the most powerful uses of action functions is the ability to adapt the ongoing trial.

Adaptation means that trial features—such as randomization ratios, sample size, trial duration, or active arms—can be modified in response to accumulated data. This reflects how many modern clinical trials are designed in practice, aiming to increase efficiency, ethical and cost balance.

In TrialSimulator, such adaptations are implemented directly within action functions. At a milestone, after retrieving the locked data and performing statistical analyses, the action function can call methods of the trial object to alter its internal state. These changes will then influence the subsequent course of the trial simulation.

Adaptations that are currently supported are summarized below:

These adaptive features allow users to simulate complex, data-driven designs where trial conduct is not static but evolves in response to accumulating evidence. At the time of writing, the vignette dedicated to trial adaptation is still under development. Detailed examples will be provided in that vignette.