--- title: "Calibrated Bayesian One-Sample and Paired t-Tests for Continuous Endpoints in Clinical Trials" author: | Riko Kelter Institute of Medical Statistics and Computational Biology Faculty of Medicine, University of Cologne Cologne, Germany date: "`r format(Sys.Date(), '%d %B %Y')`" output: rmarkdown::html_vignette: toc: true number_sections: true bibliography: references.bib vignette: > %\VignetteIndexEntry{Calibrated Bayesian One-Sample and Paired t-Tests for Continuous Endpoints in Clinical Trials} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` ```{r setup} library(bfbin2arm) # Use default controls for reproducible vignette results. # For interactive exploratory work, users may supply a faster control list. control_fast <- list( n_search = "two_stage", n_step_stage1 = 10L, rel_tol_m1 = 1e-6, subdivisions_m1 = 100L, grid_n = 121L, t_max = 6, t_max_cap = 16, root_tol = 1e-8, verify = FALSE ) control_full_fast <- list( n_search = "full_fast", rel_tol_m1 = 1e-6, subdivisions_m1 = 100L, grid_n = 121L, t_max = 6, root_tol = 1e-8, verify = FALSE, progress = TRUE ) vignette_control <- list( n_search = "full", rel_tol_m1 = 1e-9, subdivisions_m1 = 800L, grid_n = 801L, t_max = 12, t_max_cap = 40, root_tol = 1e-12, verify = TRUE, verify_rel_tol_m1 = 1e-10, verify_subdivisions_m1 = 1200L, verify_grid_n = 1201L, verify_t_max = 15, verify_root_tol = 1e-12, progress = TRUE ) ``` ```{r load-precomputed-composite-design, echo=FALSE} composite_lci_continuous <- readRDS("composite-lci-continuous.rds") composite_lci_no_pce <- readRDS("composite_lci_no_pce.rds") ``` ## Overview This vignette illustrates deterministic calibration of Bayesian one-sample and paired \(t\)-test designs for continuous clinical endpoints. The examples are prospective, trial-inspired design illustrations; they do not reproduce the original trials' analyses or numerical results. The function `calibrate_bf_ttest()` searches the requested range of sample sizes and returns the smallest feasible integer sample size. It evaluates evidence using the Bayes factor \(BF_{01}\), where smaller values favour \(H_1\) and larger values favour \(H_0\). The package supports three hypotheses: \[ H_0:\delta=0 \quad \text{versus} \quad H_1:\delta\ne0, \] \[ H_0:\delta=0 \quad \text{versus} \quad H_1:\delta>0, \] and \[ H_0:\delta\le0 \quad \text{versus} \quad H_1:\delta>0. \] Here, \(\delta\) is the standardized mean effect. For directional examples, the endpoint is coded so that positive values represent clinical benefit. ## Example 1: Two-sided point-null design This example is inspired by a phase-2 COPD study involving ivacaftor, compare @vijaykumarIvacaftorChronicObstructive2025a. We consider a hypothetical single-arm proof-of-concept study whose endpoint is change in prebronchodilator forced expiratory volume in 1 second (FEV1) percent predicted at Day 84. Suppose that a 5-percentage-point improvement is clinically relevant and that the planning standard deviation is 12.5 percentage points. The corresponding standardized effect is \[ \delta_A = \frac{5}{12.5} = 0.40. \] We use the two-sided point-null hypothesis \[ H_0:\delta=0 \quad\text{versus}\quad H_1:\delta\ne0. \] The analysis prior under \(H_1\) is \(N(0, 0.5^2)\). This prior defines the Bayes factor used at analysis; it is not the same as the design prior used to evaluate operating characteristics. The design prior under \(H_0\) is a point mass at \(\delta=0\). Under \(H_1\), we use a truncated normal design prior centred at the anticipated beneficial effect, \(TN(0.40, 0.20^2; [0,\infty))\). This represents uncertainty about the size of a beneficial effect while excluding harmful effects from the beneficial-effect planning scenario. ```{r pulmonary-two-sided, eval = FALSE} pulmonary_normal_dp <- calibrate_bf_ttest( design = "one.sample", hypothesis = "two.sided.pointnull", # Search range for the number of evaluable participants n_min = 20, n_max = 200, # Evidence thresholds: # BF01 <= 0.10 is compelling evidence for H1; # BF01 >= 3 is compelling evidence for H0. k1 = 1 / 10, k0 = 3, # Analysis prior under H1: N(0, 0.5^2) analysis_prior_sd = 0.5, # Error probability for incorrectly obtaining evidence for H1 # when the true standardized effect is exactly zero. freq_t1e = 0.05, bayes_t1e = 0.05, # Frequentist power at the clinically relevant point alternative. freq_power = list( main = list(effect = 0.40, target = 0.80) ), # Average probability of evidence for H1 under the H1 design prior. bayes_power = 0.80, # Probability of compelling evidence for H0 under the H0 design prior. # Including this argument also computes and stores the H0 evidence region. pce_h0 = 0.60, # Design priors used only for operating-characteristic calculations. design_prior_h0 = list(type = "point", effect = 0), design_prior_h1 = list( type = "truncnormal", mean = 0.40, sd = 0.20, lower = 0, upper = Inf ), control = control_full_fast ) print(pulmonary_normal_dp) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: two.sided.pointnull n*: 161 (df = 160 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: N(0, 0.5^2) family: normal location: 0 scale: 0.5 support: [-Inf, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] -Inf -2.951809 [2,] 2.951809 Inf Evidence for H0 region(s) on t-scale: lower upper [1,] -1.248312 1.248312 Constraints: constraint type freq_t1e frequentist bayes_t1e Bayesian bayes_power Bayesian pce_h0 Bayesian freq_power_main frequentist estimand estimate target P_delta=0(BF01 <= k1) 0.003634714 0.05 P_design,H0(BF01 <= k1) 0.003634714 0.05 P_design,H1(BF01 <= k1) 0.800167910 0.80 P_design,H0(BF01 >= k0) 0.786259448 0.60 P_delta=0.4(BF01 <= k1) 0.982124193 0.80 direction slack satisfied <= 0.04636529 TRUE <= 0.04636529 TRUE >= 0.00016791 TRUE >= 0.18625945 TRUE >= 0.18212419 TRUE ``` The printed result reports the selected sample size \(n^*\), the decision regions on the observed-\(t\) scale, and the attained values of every active constraint. For this configuration, the calibrated result is \(n^*=161\) (\(df=160\)); the evidence-for-\(H_1\) region is approximately \(t\le -2.95\) or \(t\ge 2.95\). ```{r, eval = FALSE} plot(pulmonary_normal_dp) ``` ```{r fig.align = "center", echo = FALSE, out.width = "100%", fig.cap = "Figure 1: Output of the plot function for a one-sample Bayesian two-sided point-null t-test design based on full calibration. Both Bayesian and frequentist power and type-I-error are calibrated jointly."} knitr::include_graphics("figures/bfbin2arm-onesample-t-test-ex1.png") ``` The frequentist type-I error is the probability, at \(\delta=0\), of obtaining \(BF_{01}\le 0.10\), that is, compelling evidence for \(H_1\) when the point null is true. The Bayesian type-I error averages that same probability over the $H_0$ design prior. Because the $H_0$ design prior here is a point mass at zero, these two values coincide. The operating-characteristic plot displays their shared curve as “Frequentist type-I error” rather than drawing two coincident curves and repeating the legend entry. The frequentist power constraint evaluates \(P_{\delta=0.40}(BF_{01}\le 0.10)\). In contrast, `bayes_power` averages \(P_{\delta}(BF_{01}\le 0.10)\) over the continuous $H_1$ design prior. Thus, the two power requirements answer different planning questions. The `pce_h0` constraint evaluates the probability of obtaining compelling evidence for the null, \(BF_{01}\ge3\), under the $H_0$ design prior. This implies that a minimum probability of obtaining a Bayes factor in favour of $H_0$ of at least size $3$ or larger is fixed, when $H_0$ is true. In this example, it also causes the function to calculate the central evidence-for-$H_0$ region. The interval is shown by the printed object and by the blue-shaded region in the decision-function plot in the top left. The decision panel displays \(\log(BF_{01})\). The orange dashed line denotes \(\log(k_1)\), so values below it support \(H_1\); the blue dashed line denotes \(\log(k_0)\), so values above it support \(H_0\). The shaded regions represent the stored decision regions: orange for evidence for \(H_1\), blue for evidence for \(H_0\), and grey for inconclusive results. The sample-size panel shows the operating characteristics throughout the search and marks the selected \(n^*\). ### A brief comparison of calibration modes The first example above calibrated both frequentist and Bayesian operating characteristics such as power and type-I-error jointly. Also, the probability of compelling evidence for $H_0$ was calibrated. Next, we compare the different calibration modes available. We can pick whatever combination we like as the calibration target, but the most prominent choices might include: - Frequentist calibration - Bayesian calibration - Hybrid calibration - Full calibration We use identical arguments including our design and analysis priors for all calibrations below, as specified in the following code chunk: ```{r, eval = FALSE} library(bfbin2arm) vignette_control <- resolve_control_bf_ttest() common_args <- list( design = "one.sample", hypothesis = "two.sided.pointnull", n_min = 20, n_max = 200, # BF01 <= 0.10: evidence for H1 # BF01 >= 3: available as a reference threshold, but no PCE constraint here k1 = 1 / 10, k0 = 3, # Analysis prior under H1: N(0, 0.5^2) analysis_prior_sd = 0.5, # H0 design prior: exact null design_prior_h0 = list(type = "point", effect = 0), # H1 design prior: uncertain beneficial effect design_prior_h1 = list( type = "truncnormal", mean = 0.40, sd = 0.20, lower = 0, upper = Inf ), control = control_full_fast ) ``` #### Frequentist calibration This design requires frequentist type-I error at most 0.05 and frequentist power at $\delta=0.40$ of at least 0.80. ```{r, eval = FALSE} pulmonary_freq <- do.call( calibrate_bf_ttest, c( common_args, list( freq_t1e = 0.05, freq_power = list( main = list(effect = 0.40, target = 0.80) ) ) ) ) print(pulmonary_freq) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: two.sided.pointnull n*: 88 (df = 87 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: N(0, 0.5^2) family: normal location: 0 scale: 0.5 support: [-Inf, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] -Inf -2.897943 [2,] 2.897943 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type estimand freq_t1e frequentist P_delta=0(BF01 <= k1) freq_power_main frequentist P_delta=0.4(BF01 <= k1) estimate target direction slack satisfied 0.004751998 0.05 <= 0.0452480024 TRUE 0.800302685 0.80 >= 0.0003026846 TRUE ``` #### Bayesian calibration This design requires Bayesian type-I error at most 0.05 and Bayesian power of at least 0.80 under the specified $H_1$ design prior. ```{r, eval = FALSE} pulmonary_bayes <- do.call( calibrate_bf_ttest, c( common_args, list( bayes_t1e = 0.05, bayes_power = 0.80 ) ) ) print(pulmonary_bayes) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: two.sided.pointnull n*: 161 (df = 160 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: N(0, 0.5^2) family: normal location: 0 scale: 0.5 support: [-Inf, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] -Inf -2.951809 [2,] 2.951809 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type estimand estimate bayes_t1e Bayesian P_design,H0(BF01 <= k1) 0.003634714 bayes_power Bayesian P_design,H1(BF01 <= k1) 0.800167910 target direction slack satisfied 0.05 <= 0.04636529 TRUE 0.80 >= 0.00016791 TRUE ``` #### Hybrid calibration This hybrid design controls type-I error at the exact null and requires average Bayesian power under the H1 design prior. ```{r, eval = FALSE} pulmonary_bayes_power_freq_t1e <- do.call( calibrate_bf_ttest, c( common_args, list( freq_t1e = 0.05, bayes_power = 0.80 ) ) ) print(pulmonary_bayes_power_freq_t1e) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: two.sided.pointnull n*: 161 (df = 160 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: N(0, 0.5^2) family: normal location: 0 scale: 0.5 support: [-Inf, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] -Inf -2.951809 [2,] 2.951809 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type estimand estimate freq_t1e frequentist P_delta=0(BF01 <= k1) 0.003634714 bayes_power Bayesian P_design,H1(BF01 <= k1) 0.800167910 target direction slack satisfied 0.05 <= 0.04636529 TRUE 0.80 >= 0.00016791 TRUE ``` #### Full calibration This activates all four error/power constraints, but deliberately does not require a minimum probability of compelling evidence for $H_0$. ```{r, eval = FALSE} pulmonary_full_no_pce <- do.call( calibrate_bf_ttest, c( common_args, list( freq_t1e = 0.05, bayes_t1e = 0.05, freq_power = list( main = list(effect = 0.40, target = 0.80) ), bayes_power = 0.80 # Deliberately omit: # pce_h0 = ... ) ) ) print(pulmonary_full_no_pce) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: two.sided.pointnull n*: 161 (df = 160 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: N(0, 0.5^2) family: normal location: 0 scale: 0.5 support: [-Inf, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] -Inf -2.951809 [2,] 2.951809 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type estimand freq_t1e frequentist P_delta=0(BF01 <= k1) bayes_t1e Bayesian P_design,H0(BF01 <= k1) bayes_power Bayesian P_design,H1(BF01 <= k1) freq_power_main frequentist P_delta=0.4(BF01 <= k1) estimate target direction slack satisfied 0.003634714 0.05 <= 0.04636529 TRUE 0.003634714 0.05 <= 0.04636529 TRUE 0.800167910 0.80 >= 0.00016791 TRUE 0.982124193 0.80 >= 0.18212419 TRUE ``` ```{r compare-calibrations, echo = FALSE, eval = FALSE} designs <- list( "Frequentist only" = pulmonary_freq, "Bayesian only" = pulmonary_bayes, "Bayesian power + frequentist type-I error" = pulmonary_bayes_power_freq_t1e, "Full calibration, no PCE" = pulmonary_full_no_pce ) active_constraints <- function(x) { tab <- x$constraint_table labels <- c( freq_t1e = "Freq. type-I error", bayes_t1e = "Bayes type-I error", bayes_power = "Bayes power", freq_power_main = "Freq. power" ) paste(unname(labels[tab$constraint]), collapse = "; ") } attained_value <- function(x, constraint) { tab <- x$constraint_table idx <- match(constraint, tab$constraint) if (is.na(idx)) { return(NA_real_) } tab$estimate[idx] } comparison_table <- data.frame( Calibration = names(designs), `n*` = vapply(designs, `[[`, integer(1), "n"), `Active constraints` = vapply(designs, active_constraints, character(1)), `Freq. type-I error` = vapply( designs, attained_value, numeric(1), constraint = "freq_t1e" ), `Bayes type-I error` = vapply( designs, attained_value, numeric(1), constraint = "bayes_t1e" ), `Freq. power at delta = 0.40` = vapply( designs, attained_value, numeric(1), constraint = "freq_power_main" ), `Bayes power` = vapply( designs, attained_value, numeric(1), constraint = "bayes_power" ), check.names = FALSE ) probability_columns <- c( "Freq. type-I error", "Bayes type-I error", "Freq. power at delta = 0.40", "Bayes power" ) comparison_table[probability_columns] <- lapply( comparison_table[probability_columns], function(x) ifelse(is.na(x), "", sprintf("%.3f", x)) ) knitr::kable( comparison_table, caption = paste( "Comparison of the four calibration strategies.", "Blank cells denote operating characteristics that were not active constraints." ), align = c("l", "r", "l", "r", "r", "r", "r") ) ``` The frequentist-only calibration selects \(n^*=88\), because it requires 80% probability of evidence for \(H_1\) only at the single planning effect \(\delta=0.40\). The Bayesian-power requirement averages this probability over the H1 design prior, which includes smaller positive effects; this produces the larger sample size \(n^*=161\). The Bayesian-only, Bayesian-power-plus-frequentist-type-I-error, and full no-PCE calibrations select the same sample size in this illustration. This occurs because the $H_0$ design prior is a point mass at \(\delta=0\), making the Bayesian and frequentist type-I error quantities numerically identical, and because the attained frequentist power at \(\delta=0.40\) already exceeds its 0.80 target at \(n=161\). No calibration includes a PCE requirement. Hence, none requires a minimum probability of compelling evidence for \(H_0\). ## Example 2: Directional point-null design Consider the same pulmonary endpoint when the confirmatory scientific question is restricted to benefit. The directional point-null test compares \(H_0:\delta=0\) with \(H_1:\delta>0\). The analysis prior under \(H_1\) is a normal distribution truncated to positive standardized effects, \(N(0, 0.5^2)\mathbb{I}(\delta>0)\). Consequently, an adverse observed change does not provide evidence for the beneficial alternative. The design prior under \(H_1\) represents uncertainty about the anticipated beneficial effect. ```{r pulmonary-directional-setup} directional_common_args <- list( design = "one.sample", hypothesis = "greater.pointnull", # Search range for the number of evaluable participants. n_min = 20, n_max = 200, # BF01 <= 0.10: compelling evidence for benefit (H1). # BF01 >= 3: available as a decision threshold, but no PCE constraint is # imposed in the calibrations below. k1 = 1 / 10, k0 = 3, # Directional H1 analysis prior: # N(0, 0.5^2), truncated to delta > 0. analysis_prior_sd = 0.5, # H0 design prior: exact null effect. design_prior_h0 = list(type = "point", effect = 0), # H1 design prior: uncertain beneficial effect. design_prior_h1 = list( type = "truncnormal", mean = 0.40, sd = 0.20, lower = 0, upper = Inf ), control = control_full_fast ) ``` ### Frequentist calibration This design controls frequentist type-I error at the exact null and requires frequentist power of at least 80% at \(\delta=0.40\). ```{r pulmonary-directional-freq, eval = FALSE} pulmonary_directional_freq <- do.call( calibrate_bf_ttest, c( directional_common_args, list( freq_t1e = 0.05, freq_power = list( main = list(effect = 0.40, target = 0.80) ) ) ) ) print(pulmonary_directional_freq) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: greater.pointnull n*: 75 (df = 74 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: TN(0, 0.5^2; [0, Inf)) family: normal location: 0 scale: 0.5 support: [0, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] 2.601071 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type estimand freq_t1e frequentist P_delta=0(BF01 <= k1) freq_power_main frequentist P_delta=0.4(BF01 <= k1) estimate target direction slack satisfied 0.005608335 0.05 <= 0.044391665 TRUE 0.803068443 0.80 >= 0.003068443 TRUE ``` We see that the number of patients decreases compared to the two-sided test. We can plot the design: ```{r, eval = FALSE} plot(pulmonary_directional_freq) ``` ```{r fig.align = "center", echo = FALSE, out.width = "100%", fig.cap = "Figure 2: Output of the plot function for a one-sample Bayesian t-test design based on frequentist calibration."} knitr::include_graphics("figures/bfbin2arm-onesample-t-test-freq.png") ``` Figure 2 shows that the shape of the decision function now reflects the one-sided test. For increasingly large t-statistic, the Bayes factor will reject $H_0$. Note that the top right panel in Figure 2 shows Bayesian power (as the design prior under $H_1$ is specified in the common parameters above), but only frequentist power and type-I-error are used for calibrating the required sample size. ### Bayesian calibration This design controls the $H_0$-design-prior average probability of incorrectly obtaining compelling evidence for benefit and requires 80% average Bayesian power under the $H_1$ design prior. ```{r pulmonary-directional-bayes, eval = FALSE} pulmonary_directional_bayes <- do.call( calibrate_bf_ttest, c( directional_common_args, list( bayes_t1e = 0.05, bayes_power = 0.80 ) ) ) print(pulmonary_directional_bayes) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: greater.pointnull n*: 134 (df = 133 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: TN(0, 0.5^2; [0, Inf)) family: normal location: 0 scale: 0.5 support: [0, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] 2.666093 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type bayes_t1e Bayesian bayes_power Bayesian estimand estimate target P_design,H0(BF01 <= k1) 0.00431281 0.05 P_design,H1(BF01 <= k1) 0.80025788 0.80 direction slack satisfied <= 0.0456871901 TRUE >= 0.0002578833 TRUE ``` Since the Bayesian type-I-error under $H_0:\delta = 0$ is computed under a point-prior, the Bayesian type-I-error rate is identical to the frequentist one. This is the reason that the plot shows frequentist type-I-error (which usually is more conservative), even though the calibration uses Bayesian type-I-error (and power). ```{r, eval = FALSE} plot(pulmonary_directional_bayes) ``` ```{r fig.align = "center", echo = FALSE, out.width = "100%", fig.cap = "Figure 3: Output of the plot function for a one-sample Bayesian t-test design based on Bayesian calibration."} knitr::include_graphics("figures/bfbin2arm-onesample-t-test-bayes.png") ``` ### Hybrid calibration This hybrid calibration uses a frequentist type-I error requirement at \(\delta=0\) and a Bayesian power requirement averaged over the $H_1$ design prior. ```{r pulmonary-directional-hybrid, eval = FALSE} pulmonary_directional_bayes_power_freq_t1e <- do.call( calibrate_bf_ttest, c( directional_common_args, list( freq_t1e = 0.05, bayes_power = 0.80 ) ) ) print(pulmonary_directional_bayes_power_freq_t1e) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: greater.pointnull n*: 134 (df = 133 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: TN(0, 0.5^2; [0, Inf)) family: normal location: 0 scale: 0.5 support: [0, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] 2.666093 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type freq_t1e frequentist bayes_power Bayesian estimand estimate target P_delta=0(BF01 <= k1) 0.00431281 0.05 P_design,H1(BF01 <= k1) 0.80025788 0.80 direction slack satisfied <= 0.0456871901 TRUE >= 0.0002578833 TRUE ``` ### Full calibration without PCE This version activates frequentist and Bayesian type-I error requirements, as well as frequentist and Bayesian power requirements. It does not impose a probability-of-compelling-evidence (PCE) requirement for \(H_0\). ```{r pulmonary-directional-full, eval = FALSE} pulmonary_directional_full_no_pce <- do.call( calibrate_bf_ttest, c( directional_common_args, list( freq_t1e = 0.05, bayes_t1e = 0.05, freq_power = list( main = list(effect = 0.40, target = 0.80) ), bayes_power = 0.80 ) ) ) print(pulmonary_directional_full_no_pce) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: greater.pointnull n*: 134 (df = 133 ) Thresholds: k1 = 0.1 , k0 = 3 Analysis prior under H0: Point prior: delta = 0 Analysis prior under H1: Prior: TN(0, 0.5^2; [0, Inf)) family: normal location: 0 scale: 0.5 support: [0, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] 2.666093 Inf Evidence for H0 region(s) on t-scale: none Constraints: constraint type freq_t1e frequentist bayes_t1e Bayesian bayes_power Bayesian freq_power_main frequentist estimand estimate target P_delta=0(BF01 <= k1) 0.00431281 0.05 P_design,H0(BF01 <= k1) 0.00431281 0.05 P_design,H1(BF01 <= k1) 0.80025788 0.80 P_delta=0.4(BF01 <= k1) 0.97401497 0.80 direction slack satisfied <= 0.0456871901 TRUE <= 0.0456871901 TRUE >= 0.0002578833 TRUE >= 0.1740149710 TRUE ``` We see that all calibrations except the frequentit one require $n^{\star}=134$ patients. Frequentist calibration under the much more optimistic point prior at $\delta = 0.4$ requires solely $n^{\star}=75$ patients. The point-null versus greater directional test shows that the directional Bayes factor is asymmetric in the observed \(t\) statistic, compare the top left panel in Figure 2. The region providing evidence for \(H_1\) should therefore occur in the positive tail only. The implementation determines decision regions by finding the relevant Bayes-factor roots and classifying the resulting intervals rather than imposing a symmetric decision geometry. ## Example 3: Directional composite-null design This example is inspired by an 8-week open-label, multicentre study of inhaled tobramycin in people with cystic fibrosis and *Pseudomonas aeruginosa* infection, compare @sutharsan8WeekOpenlabel2020a. The published study assessed pulmonary outcomes including forced expiratory volume in 1 second (FEV1) and lung-clearance index (LCI). We consider a hypothetical future single-arm proof-of-concept study. Let the endpoint be improvement in LCI from baseline to Week 8, coded as baseline minus follow-up LCI. Thus, a positive value denotes clinical improvement. The scientific question is whether the intervention provides evidence of benefit rather than merely evidence against an exact zero mean change: \[ H_0:\delta\leq0 \quad\text{versus}\quad H_1:\delta>0. \] The null-side hypothesis includes both no improvement and deterioration. This is appropriate when a benefit claim should be made only when the study yields compelling evidence for an improvement. ```{r, eval = FALSE} composite_lci_continuous <- calibrate_bf_ttest( design = "one.sample", hypothesis = "greater.composite", n_min = 20, n_max = 120, k1 = 1 / 20, k0 = 3, analysis_prior_h0 = make_analysis_prior_less_equal(sd = 0.5), analysis_prior_h1 = make_analysis_prior_greater(sd = 0.5), freq_t1e = 0.05, freq_power = list(main = list(effect = 0.40, target = 0.80)), bayes_t1e = 0.05, bayes_power = 0.80, pce_h0 = 0.60, design_prior_h0 = list( type = "truncnormal", mean = -0.10, sd = 0.25, lower = -Inf, upper = 0 ), design_prior_h1 = list( type = "truncnormal", mean = 0.40, sd = 0.20, lower = 0, upper = Inf ), control = fast_control ) composite_lci_continuous ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: one.sample Hypothesis: greater.composite n*: 65 (df = 64 ) Thresholds: k1 = 0.05 , k0 = 3 Analysis prior under H0: Prior: TN(0, 0.5^2; (-Inf, 0]) family: normal location: 0 scale: 0.5 support: [-Inf, 0] Analysis prior under H1: Prior: TN(0, 0.5^2; [0, Inf)) family: normal location: 0 scale: 0.5 support: [0, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] 1.733227 Inf Evidence for H0 region(s) on t-scale: lower upper [1,] -Inf -0.6935961 Constraints: constraint type estimand estimate freq_t1e frequentist P_delta=0(BF01 <= k1) 0.043935585 bayes_t1e Bayesian P_design,H0(BF01 <= k1) 0.005294749 bayes_power Bayesian P_design,H1(BF01 <= k1) 0.802264572 pce_h0 Bayesian P_design,H0(BF01 >= k0) 0.758721651 freq_power_main frequentist P_delta=0.4(BF01 <= k1) 0.930717800 target direction slack satisfied 0.05 <= 0.006064415 TRUE 0.05 <= 0.044705251 TRUE 0.80 >= 0.002264572 TRUE 0.60 >= 0.158721651 TRUE 0.80 >= 0.130717800 TRUE ``` ```{r, eval = FALSE} plot(composite_lci_continuous) ``` ```{r fig.align = "center", echo = FALSE, out.width = "100%", fig.cap = "Figure 3: Output of the plot function for a one-sample Bayesian composite-null versus directional t-test design based on full calibration. Both Bayesian and frequentist power and type-I-error are calibrated jointly. Also, a 60% threshold on the probability of compelling evidence for the null hypothesis is guaranteed."} knitr::include_graphics("figures/bfbin2am-onesample-t-test-ex3.png") ``` As a sidenote, single-arm change-from-baseline evidence is vulnerable to within-person variability, natural history, regression to the mean, and measurement error; it does not itself establish a causal treatment effect, in contrast to two-arm change-from-baseline evidence. However, the composite-null formulation clarifies the evidential decision rule. What it cannot achieve, however, it to remove the limitations of a non-comparative design. External controls, randomization, or other design features may be necessary for a confirmatory efficacy claim. We study this example nevertheless, because a directional composite-null design is widely used in clinical trials. ## Comparison with a conventional frequentist design The conventional comparator is a one-sided one-sample t-test, \[ H_0: \delta \leq 0 \qquad \text{versus} \qquad H_1: \delta > 0, \] designed for a type-I error rate of 0.05 and 80% power at the fixed alternative \(\delta = 0.40\). We also consider a Bayes-factor design that retains the frequentist type-I-error and power requirements but does not require compelling evidence for the H0 region. This separates the sample-size implication of requiring strong evidence for H1 from the additional requirement to obtain compelling H0 evidence. ```{r} ## Conventional frequentist one-sided one-sample t-test freq_design <- power.t.test( delta = 0.40, sd = 1, sig.level = 0.05, power = 0.80, type = "one.sample", alternative = "one.sided" ) freq_n <- ceiling(freq_design$n) ``` ```{r sample-size-comparisons, eval = FALSE} ## Bayes-factor design: H1 evidence only, no PCE requirement. composite_lci_no_pce <- calibrate_bf_ttest( design = "one.sample", hypothesis = "greater.composite", n_min = 20, n_max = 120, k1 = 1 / 20, k0 = 3, analysis_prior_h0 = make_analysis_prior_less_equal(sd = 0.5), analysis_prior_h1 = make_analysis_prior_greater(sd = 0.5), freq_t1e = 0.05, freq_power = list(main = list(effect = 0.40, target = 0.80)), design_prior_h0 = list( type = "truncnormal", mean = -0.10, sd = 0.25, lower = -Inf, upper = 0 ), design_prior_h1 = list( type = "truncnormal", mean = 0.40, sd = 0.20, lower = 0, upper = Inf ), control = fast_control ) ``` ```{r} sample_size_comparison <- data.frame( design = c( "Frequentist one-sample t-test", "Bayes-factor design: H1 evidence only", "Bayes-factor design: H1 and H0 evidence" ), type_I_error_control = c( "One-sided alpha = 0.05", "P(delta = 0; BF01 <= 1/20) <= 0.05", "P(delta = 0; BF01 <= 1/20) <= 0.05" ), power_requirement = c( "Power(delta = 0.40) >= 0.80", "P(delta = 0.40; BF01 <= 1/20) >= 0.80", "P(delta = 0.40; BF01 <= 1/20) >= 0.80" ), H0_evidence_requirement = c( "None", "None", "P_design,H0(BF01 >= 3) >= 0.60" ), sample_size = c( freq_n, composite_lci_no_pce$n, composite_lci_continuous$n ), stringsAsFactors = FALSE ) knitr::kable( sample_size_comparison, col.names = c( "Design", "Type-I-error control", "Power requirement", "H0-evidence requirement", "Required n" ), caption = "Sample-size comparison under a one-sided standardized effect of 0.40." ) ``` The conventional frequentist calculation requires `r freq_n` patients. The BF design without a probability-of-compelling-evidence (PCE) requirement requires `r composite_lci_no_pce$n` patients. Its slightly larger sample size reflects the use of the more stringent evidential criterion \(BF_{01} \leq 1/20\), rather than a conventional one-sided significance threshold. Adding the requirement that the design yield compelling $H_0$ evidence (\(BF_{01} \geq 3\)) with probability at least 0.60 under the $H_0$ design prior increases the required sample size to `r composite_lci_continuous$n`. Thus, the PCE requirement adds `r composite_lci_continuous$n - composite_lci_no_pce$n` patients beyond the $H_1$-evidence-only BF design, and the complete calibrated design adds `r composite_lci_continuous$n - freq_n` patients beyond the conventional frequentist design. ## Example 4: Paired one-sample design This example is inspired by the Healthy BP study, which compared a dietitian-led telehealth lifestyle intervention with a self-directed intervention. Its primary endpoint was change in 24-hour ambulatory systolic blood pressure from baseline to 12 weeks, see @taherTelehealthSelfdirectedLifestyle2021. Here, we consider a hypothetical future single-group implementation study with measurements at the same two time points. This is a prospective design illustration, not a reproduction of the original randomized trial or its between-group analysis. All effect sizes, prior distributions, evidence thresholds, and calibration targets below are illustrative planning assumptions rather than quantities attributed to that trial. ### Paired outcome and hypothesis Define the participant-level outcome as \[ D_i = X_{i,\mathrm{baseline}} - X_{i,\mathrm{12\ weeks}}, \] so that a positive difference represents a beneficial reduction in systolic blood pressure. Let \[ \mu_D = \mathbb{E}(D_i), \qquad \sigma_D = \operatorname{SD}(D_i), \qquad \delta = \frac{\mu_D}{\sigma_D}. \] A clinically relevant reduction of 5 mmHg and an assumed planning standard deviation of paired differences of 12.5 mmHg imply \[ \delta_A = \frac{5}{12.5} = 0.40. \] The standard deviation in this calculation is the standard deviation of the within-participant differences, not the standard deviation of baseline or follow-up blood pressure considered separately. We use the directional composite-null hypotheses \[ H_0:\delta\leq0 \qquad\text{versus}\qquad H_1:\delta>0. \] Thus, the null-side hypothesis includes both no mean reduction and a mean increase in blood pressure. For a paired design, \(n\) is the number of participants with evaluable baseline and follow-up measurements. The test statistic is \[ T = \frac{\sqrt{n}\,\bar D}{S_D}, \qquad T\mid\delta \sim t_{n-1}(\sqrt{n}\delta). \] The statistical engine is therefore the same as for a one-sample test of the participant-level differences. ### Priors, evidence thresholds, and calibration targets We use normal analysis priors with scale 0.5, truncated and normalized on the respective hypothesis regions: \[ \delta\mid H_0 \sim N(0,0.5^2)\big|_{(-\infty,0]}, \qquad \delta\mid H_1 \sim N(0,0.5^2)\big|_{(0,\infty)}. \] These analysis priors define the Bayes factor. For planning, we use separate design priors: \[ \pi_{d,0} = N(-0.10,0.25^2)\big|_{(-\infty,0]}, \qquad \pi_{d,1} = N(0.40,0.20^2)\big|_{(0,\infty)}. \] The evidence thresholds are fixed in advance: - $BF_{01}\leq1/20$: compelling evidence for a beneficial reduction; - $BF_{01}\geq3$: compelling evidence for the null-side hypothesis; - otherwise: an inconclusive result. We jointly require: - pointwise frequentist type-I error at \(\delta=0\) at most 0.05; - frequentist power at \(\delta=0.40\) at least 0.80; - Bayesian type-I error under the null-side design prior at most 0.05; - Bayesian power under the beneficial-effect design prior at least 0.80; - probability of compelling null-side evidence under the null-side design prior at least 0.60. ```{r paired-bp-calibration, eval = FALSE} paired_bp <- calibrate_bf_ttest( design = "paired", hypothesis = "greater.composite", # Number of evaluable participant pairs. n_min = 20, n_max = 120, # Prespecified, fixed evidence thresholds. k1 = 1 / 20, k0 = 3, # Default composite-null analysis priors: # N(0, 0.5^2), truncated to the respective half-lines. analysis_prior_sd = 0.5, freq_t1e = 0.05, freq_power = list( main = list(effect = 0.40, target = 0.80) ), bayes_t1e = 0.05, bayes_power = 0.80, pce_h0 = 0.60, design_prior_h0 = list( type = "truncnormal", mean = -0.10, sd = 0.25, lower = -Inf, upper = 0 ), design_prior_h1 = list( type = "truncnormal", mean = 0.40, sd = 0.20, lower = 0, upper = Inf ), control = control_full_fast ) print(paired_bp) ``` ``` Calibrated Bayesian t-test design --------------------------------- Design: paired Hypothesis: greater.composite n*: 65 (df = 64 ) Thresholds: k1 = 0.05 , k0 = 3 Analysis prior under H0: Prior: TN(0, 0.5^2; (-Inf, 0]) family: normal location: 0 scale: 0.5 support: [-Inf, 0] Analysis prior under H1: Prior: TN(0, 0.5^2; [0, Inf)) family: normal location: 0 scale: 0.5 support: [0, Inf] Evidence for H1 region(s) on t-scale: lower upper [1,] 1.733227 Inf Evidence for H0 region(s) on t-scale: lower upper [1,] -Inf -0.6935961 Constraints: constraint type estimand estimate freq_t1e frequentist P_delta=0(BF01 <= k1) 0.043935585 bayes_t1e Bayesian P_design,H0(BF01 <= k1) 0.005294749 bayes_power Bayesian P_design,H1(BF01 <= k1) 0.802264572 pce_h0 Bayesian P_design,H0(BF01 >= k0) 0.758721651 freq_power_main frequentist P_delta=0.4(BF01 <= k1) 0.930717800 target direction slack satisfied 0.05 <= 0.006064415 TRUE 0.05 <= 0.044705251 TRUE 0.80 >= 0.002264572 TRUE 0.60 >= 0.158721651 TRUE 0.80 >= 0.130717800 TRUE ``` ### Calibrated design The selected sample size is \(n^\star = 65\) evaluable participant pairs, with $64$ degrees of freedom. Thus, the design requires at least $65$ participants contributing both measurements, not $2\cdot 65=130$ independently sampled participants. ### Interpretation and relation to the one-sample example This example uses the same standardized-effect priors, fixed evidence thresholds, and calibration requirements as the composite-null one-sample example. Consequently, with identical numerical controls, the paired and one-sample calculations represent the same mathematical calibration problem. What changes is the interpretation of the outcome and sample size: here the observations are participant-level differences and \(n\) counts evaluable pairs. Repeated baseline and follow-up measurements are not independent observations. Their dependence is incorporated through the standard deviation of the paired differences. If planning information is available only for the marginal standard deviations and within-participant correlation, then \[ \sigma_D^2 = \sigma_{\mathrm{baseline}}^2 + \sigma_{\mathrm{follow\text{-}up}}^2 - 2\rho\, \sigma_{\mathrm{baseline}} \sigma_{\mathrm{follow\text{-}up}}. \] The assumed value \(\sigma_D=12.5\) mmHg should therefore be justified using relevant paired data or examined in sensitivity analyses. Finally, the calibrated sample size refers to evaluable pairs. Recruitment inflation for missing follow-up measurements should be considered separately. Neither such inflation nor the paired analysis itself resolves bias arising from informative missingness or the causal limitations of a single-group study. ## Interpretation `calibrate_bf_ttest()` returns the smallest sample size in the requested range that satisfies the activated constraints. The returned object includes the Bayes-factor evidence regions, attained operating characteristics, constraint table, and numerical controls used for the calculation. The examples are illustrations of a design procedure. They do not imply that a Bayesian design automatically requires fewer participants than a conventional design; any difference follows from the chosen evidence thresholds and jointly calibrated operating-characteristic requirements. ## References