Package odr

The costs of sampling each additional unit in multilevel experimental studies vary across levels of hierarchy and treatment conditions due to the hierarchical sampling and the delivery of treatment. This package is a tool to optimize the designs of multilevel experimental studies such that the variances of treatment effects are minimized under fixed budget and cost structure, or the budget is mimimized to achieve same level design precision or statistical power. The optimal sample allocation or optimal design parameters include

For each type of multilevel experimental studies, there is additional number(s) and/or letter(s) to be added to the general function name except for the re function. For example, the two functions for two-level cluster randomized trials (CRTs) with treatment randomization at the level-2 are od.2 and power.2; the two functions for 2-level multisite cluster randomized trials with treatment randomization at level-1 are od.2m1 and power.2m1

1. Function od

Given cost structure (i.e., the costs of sampling each unit at different levels and treatment conditions), this function solves the optimal sample allocation with and without constraints.

To solve the optimal sample allocation of two-level CRTs, we need the following information

icc: intraclass correlation coefficient in the population or in each treatment condtion
r12: the level-1 outcome variance explained by covariates in the population or in each treatment condtion
r22: the level-2 outcome variance explained by covariates in the population or in each treatment condtion
c1: the cost of sampling each additional level-1 unit in control condition
c2: the cost of sampling each additional level-2 unit in control condition
c1t: the cost of sampling each additional level-1 unit in treatment condition
c2t: the cost of sampling each additional level-2 unit in treatment condition
m: total fixed budget used to plot the variance curves, default is the cost of sampling 60 level-2 units across treatment conditions.

1.1 Examples

library(odr)

 # unconstrained optimal design
myod1 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50, 
              varlim = c(0.01, 0.02))

## The optimal level-1 sample size per level-2 unit (n) is 8.878572.
## The optimal proportion of level-2 units in treatment (p) is 0.326828.

 # the function by default prints messages of output and plots the variance curves; one can turn off message and specify one or no plot; default m value is the total costs of sampling 60 level-2 units across treatment conditions, m can be explicitly specified.
 # myod1$out for output; myod1$par for parameters used in the calculation.

 # constrained optimal design with n = 20
myod2 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50,
              plot.by = list(p = "p"), n = 20, varlim = c(0.005, 0.030))

## The constrained level-1 sample size per level-2 unit (n) is 20.
## The optimal proportion of level-2 units in treatment (p) is 0.3740667.

 # myod2$out for output; myod2$par for parameters used in the calculation.

 # constrained optimal design with p = 0.5
myod3 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50, 
             p = 0.5, varlim = c(0.005, 0.020))

## The optimal level-1 sample size per level-2 unit (n) is 10.48809.
## The constrained proportion of level-2 units in treatment (p) is 0.5.

 # myod3$out for output; myod3$par for parameters used in the calculation.

 # constrained n and p, no calculation performed
myod4 <- od.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50,
              plots = FALSE, n = 20, p = 0.5, varlim = c(0.005, 0.025))

## ===============================
## Both p and n are constrained, there is no calculation from other parameters.
## ===============================
## The constrained level-1 sample size per level-2 unit (n) is 20.
## The constrained proportion of level-2 units in treatment (p) is 0.5.

1.2 Examples for other types of trials

Please see examples in corresponding fuctions (e.g., od.3 and od.4).

# ?od.3
# ?od.4

2. Function re

Calculate the relative efficiency (RE) of two designs, this function uses the returns from od function

2.1 Examples

Based on above examples, calculate the relative efficiency

# relative efficiency (RE) of constrainted design comparing with the unconstrained optimal one
myre <- re(od = myod1, subod= myod2)

## The relative efficiency (RE) of the two two-level CRTs is 0.8790305.

myre$out # RE = 0.88

## [1] 0.8790305

# relative efficiency (RE) constrainted design comparing with the unconstrained optimal one
myre <- re(od = myod1, subod= myod3)

## The relative efficiency (RE) of the two two-level CRTs is 0.8975086.

myre$out # RE = 0.90

## [1] 0.8975086

# relative efficiency (RE) constrainted design comparing with the unconstrained optimal one
myre <- re(od = myod1, subod= myod4)

## The relative efficiency (RE) of the two two-level CRTs is 0.8266527.

myre$out # RE = 0.83

## [1] 0.8266527

2.2 Examples for other types of trials

Please see examples in corresponding fuctions (e.g., od.3 and od.4).

# ?od.3
# ?od.4

3. Function power

This function performs the calculation directly from a cost-analysis perspective. For example, this function can provide statistical power calculation under fixed budget and cost structure, MDES calculation under fixed buget and cost structure with desired power level, budget calculation with desired power to detect the treatment effect. This function also provides conventional statistical power analyses of multilevel experimental studies (e.g., power calculation, minimum detectable effect size (MDES) calculation, required sample size calculation).

This function provides power analyses with cost model on With default (i.e., cost.model = T), one of ‘power’, ‘m’, and ‘es’ must be NULL. For example, if ‘power’ is NULL, the function calculates statistical power of treatment effect under fixed budget and cost structure; if ‘es’ is NULL, the fuction calculate minimum detectable effect size (i.e., es) under fixed budget and desired power level; if ‘m’ is NULL, the function calculate required budget to achieve desired power level to detect a treatment effect. When cost model is off (i.e., cost.model = F), this function provides conventional power analyses that can calculate statistical power, MDES, or required sample size.

3.1 Examples of power analyses accommodating costs and budget (cost.model = T)

Required budget for desired power

Required budget calculation

mym <- power.2(expr = myod1, es = 0.3, q = 1, power = 0.8)
# mym$out  # m =1702

Effects on required budget when designs depart from the optimal one

figure <- par(mfrow = c(1, 2))
budget <- NULL
nrange <- c(2:50)
for (n in nrange)
  budget <- c(budget, power.2(expr = myod1, constraint = list (n = n), es = 0.3, q = 1, power = 0.8)$out$m)
plot(nrange, budget, type = "l", lty = 1, xlim = c(0, 50), ylim = c(1500, 3500),
     xlab = "Level-1 sample size: n", ylab = "Budget", main = "", col = "black")
 abline(v = 9, lty = 2, col = "Blue")
 
budget <- NULL
prange <- seq(0.05, 0.95, by = 0.005)
for (p in prange)
  budget <- c(budget, power.2(expr = myod1, constraint = list (p = p), es = 0.3, q = 1, power = 0.8)$out$m)
plot(prange, budget, type = "l", lty = 1, xlim = c(0, 1), ylim = c(1500, 7000),
     xlab = "Porportion groups in treatment: p", ylab = "Budget", main = "", col = "black")
 abline(v = 0.33, lty = 2, col = "Blue")

par(figure)

Statistical power under fixed budget

Power calculation

mypower <- power.2(expr = myod1, q = 1, es = 0.3, m = 1702)
# mypower$out  # power = 0.80

Effects on power when designs depart from the optimal one

figure <- par(mfrow = c (1, 2))
pwr <- NULL
nrange <- c(2:50)
for (n in nrange)
  pwr <- c(pwr, power.2(expr = myod1, constraint = list (n = n), es = 0.3, q = 1, m = 1702)$out)
plot(nrange, pwr, type = "l", lty = 1, xlim = c(0, 50), ylim = c(0.4, 0.9),
     xlab = "Level-1 sample size: n", ylab = "Power", main = "", col = "black")
 abline(v = 9, lty = 2, col = "Blue")
 
pwr <- NULL
prange <- seq(0.05, 0.95, by = 0.005)
for (p in prange)
  pwr <- c(pwr, power.2(expr = myod1, constraint = list (p = p), es = 0.3, q = 1, m = 1702)$out)
plot(prange, pwr, type = "l", lty = 1, xlim = c(0, 1), ylim = c(0.1, 0.9),
     xlab = "Porportion groups in treatment: p", ylab = "Power",  main = "", col = "black")
 abline(v = 0.33, lty = 2, col = "Blue")

 par(figure)

MDES under fixed budget

MDES calculation

mymdes <- power.2(expr = myod1, q = 1, power = 0.80, m = 1702)
# mymdes$out  # MDES = 0.30

Effects on MDES when designs depart from the optimal one

figure <- par(mfrow = c (1, 2))
MDES <- NULL
nrange <- c(2:50)
for (n in nrange)
  MDES <- c(MDES, power.2(expr = myod1, constraint = list (n = n), power = 0.8, q = 1, m = 1702)$out)
plot(nrange, MDES, type = "l", lty = 1, xlim = c(0, 50), ylim = c(0.3, 0.8),
     xlab = "Level-1 sample size: n", ylab = "MDES", main = "", col = "black")
 abline(v = 9, lty = 2, col = "Blue")
 
MDES <- NULL
prange <- seq(0.05, 0.95, by = 0.005)
for (p in prange)
  MDES <- c(MDES, power.2(expr = myod1, constraint = list (p = p), power = 0.8, q = 1, m = 1702)$out)
plot(prange, MDES, type = "l", lty = 1, xlim = c(0, 1), ylim = c(0.3, 0.8),
     xlab = "Porportion groups in treatment: p", ylab = "MDES", main = "", col = "black")
 abline(v = 0.33, lty = 2, col = "Blue")

 par(figure)

3.2 Examples of conventional power analyses (cost.model = FALSE)

# Required level-2 sample size calculation
myJ <- power.2(cost.model = FALSE, expr = myod1, es = 0.3, q = 1, power = 0.8)
# above experssion takes parameters and outputs from od.2 function. Equivalently, each parameter can be explicitly specified.
# myJ <- power.2(icc = 0.2, r12 = 0.5, r22 = 0.5, c1 = 1, c2 = 5, c1t = 1, c2t = 50,
#                     cost.model = FALSE, n = 9, p = 0.33, es = 0.3, q = 1, power = 0.8)
myJ$out  # J = 59

## $J
## [1] 58.99295

# Power calculation
mypower1 <- power.2(cost.model = FALSE, expr = myod1, J = 59, es = 0.3, q = 1)
mypower1$out  # power = 0.80

## $power
## [1] 0.8000486

# MDES calculation
mymdes1 <- power.2(cost.model = FALSE, expr = myod1, J = 59, power = 0.8, q = 1)
mymdes1$out  # es = 0.30

## $es
## [1] 0.2999819

3.3 Examples of conventional power plots

figure <- par(mfrow = c (1, 2))
pwr <- NULL
mrange <- c(300:3000)
for (m in mrange)
  pwr <- c(pwr, power.2(expr = myod1, es = 0.3, q = 1, m = m)$out)
plot(mrange, pwr, type = "l", lty = 1, xlim = c(300, 3000), ylim = c(0, 1),
     xlab = "Budget", ylab = "Power", main = "", col = "black")
 abline(v = 1702, lty = 2, col = "Blue")
 
pwr <- NULL
Jrange <- c(4:100)
for (J in Jrange)
  pwr <- c(pwr, power.2(expr = myod1, cost.model = FALSE, es = 0.3, q = 1, J = J)$out)
plot(Jrange, pwr, type = "l", lty = 1, xlim = c(4, 100), ylim = c(0, 1),
     xlab = "Level-2 sample size: J", ylab = "Power", main = "", col = "black")
 abline(v = 59, lty = 2, col = "Blue")

par(figure)

3.4 Examples for other types of trials

Please see examples in corresponding fuctions (e.g., power.3 and power.4).

# ?power.3
# ?power.4

Package ‘odr’

Zuchao Shen

2018-01-03