agree_test

In the simplest scenario, a study may be conducted to compare one measure (e.g., x) and another (e.g., y). In this scenario each pair of observations (x and y) are independent; meaning that each pair represents one subject/participant. In most cases we have a degree of agreement that we would deem adequate. This may constitute a hypothesis wherein you may believe the agreement between two measurements is within a certain limit (limits of agreement). If this is the goal then the agree_test function is what you need to use in this package.

The data for the two measurements are put into the x and y arguments. If there is a hypothesized limit of agreement then this can be set with the delta argument (this is optional). Next, the limit of agreement can be set with the agree.level and the confidence level (\(1-\alpha\)). Once those are set the analysis can be run. Please note, this package has pre-loaded data from the Zou 2013 paper. While data does not conform the assumptions of the test it can be used to test out many of the functions in this package. Since there isn’t an a priori hypothesis I will not declare a delta argument, but I will estimate the 95% confidence intervals for 80% limits of agreement.

a1 = agree_test(x = reps$x,
                y = reps$y,
                agree.level = .8)
#> Warning: `agree_test()` was deprecated in SimplyAgree 0.2.0.
#> ℹ Please use `agreement_limit()` instead.
#> This warning is displayed once every 8 hours.
#> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
#> generated.

We can then print the general results. These results include the general parameters of the analysis up top, then the results of the Shieh exact test for agreement (no conclusion is included due to the lack of a delta argument being set). Then the limits of agreement, with confidence limits, are included. Lastly, Lin’s Concordance Correlation Coefficient, another measure of agreement, is also included.

print(a1)
#> Limit of Agreement = 80%
#> 
#> ###- Shieh Results -###
#> Exact 90% C.I.  [-1.512, 2.3887]
#> Hypothesis Test: No Hypothesis Test
#> 
#> ###- Bland-Altman Limits of Agreement (LoA) -###
#>           Estimate Lower CI Upper CI CI Level
#> Bias        0.4383  -0.1669   1.0436     0.95
#> Lower LoA  -1.1214  -1.8037  -0.4391     0.90
#> Upper LoA   1.9980   1.3157   2.6803     0.90
#> 
#> ###- Concordance Correlation Coefficient (CCC) -###
#> CCC: 0.4791, 95% C.I. [0.1276, 0.7237]

Next, we can use the generic plot function to produce visualizations of agreement. This includes the Bland-Altman plot (type = 1) and a line-of-identity plot (type = 2).

plot(a1, type = 1)

plot(a1, type = 2) 

Calculations in for Simple Agreement

agree_reps

This function is for cases where the underlying values do not vary within subjects. This can be considered cases where replicate measure may be taken. For example, a researcher may want to compare the performance of two ELISA assays where measurements are taken in duplicate/triplicate.

So, for this function you will have to provide the data frame object with the data argument and the names of the columns containing the first (x argument) and second (y argument) must then be provided. An additional column indicating the subject identifier (id) must also be provided. Again, if there is a hypothesized agreement limit then this could be provided with the delta argument.

a2 = agree_reps(x = "x",
                y = "y",
                id = "id",
                data = reps,
                agree.level = .8)
#> Warning: `agree_reps()` was deprecated in SimplyAgree 0.2.0.
#> ℹ Please use `agreement_limit()` instead.
#> This warning is displayed once every 8 hours.
#> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
#> generated.

The results can then be printed. The printing format is very similar to agree_test, but notice that 1) the hypothesis test is based on the limits of agreement (MOVER method), 2) the Concordance Correlation Coefficient is calculated via the U-statistics method, 3) the Shieh TOST results are missing because they cannot be estimated for this type of design.

print(a2)
#> Limit of Agreement = 80%
#> Replicate Data Points (true value does not vary)
#> 
#> Hypothesis Test: No Hypothesis Test
#> 
#> ###- Bland-Altman Limits of Agreement (LoA) -###
#>           Estimate Lower CI Upper CI CI Level
#> Bias        0.7152  -0.6667   2.0971     0.95
#> Lower LoA  -1.2117  -4.7970   0.1054     0.90
#> Upper LoA   2.6421   1.3250   6.2274     0.90

plot(a2, type = 1)

plot(a2, type = 2)

agree_nest

This function is for cases where the underlying values may vary within subjects. This can be considered cases where there are distinct pairs of data wherein data is collected in different times/conditions within each subject. An example would be measuring blood pressure on two different devices on many people at different time points/days.

The function works almost identically to agree_reps but the underlying calculations are different

a3 = agree_nest(x = "x",
                y = "y",
                id = "id",
                data = reps,
                agree.level = .8)
#> Warning: `agree_nest()` was deprecated in SimplyAgree 0.2.0.
#> ℹ Please use `agreement_limit()` instead.
#> This warning is displayed once every 8 hours.
#> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
#> generated.

The printed results (and plots) are very similar to agree_reps. However, the CCC result now has a warning because the calculation in this scenario may not be entirely appropriate given the nature of the data.

print(a3)
#> Limit of Agreement = 80%
#> Nested Data Points (true value may vary)
#> 
#> Hypothesis Test: No Hypothesis Test
#> 
#> ###- Bland-Altman Limits of Agreement (LoA) -###
#>           Estimate Lower CI Upper CI CI Level
#> Bias        0.7101  -0.6824   2.1026     0.95
#> Lower LoA  -1.1626  -4.8172   0.1811     0.90
#> Upper LoA   2.5828   1.2390   6.2374     0.90

plot(a3, type = 1)

plot(a3, type = 2)

Calculations for agree_reps & agree_nest

All the calculations for the limits of agreement in these two functions can be found in the article by Zou (2011).

agree_nest LoA

Step 1: Calculate Individual Subject Means and Variances

\[ \bar x_i = \frac{1}{n_{xi}} \Sigma_{j=1}^{n_{xi}} x_{ij} \] \[ \bar y_i = \frac{1}{n_{yi}} \Sigma_{j=1}^{n_{yi}} y_{ij} \] \[ \bar d_i = \bar x_i - \bar y_i \]

\[ \bar d = \Sigma_{i=1}^{n}\frac{d_i}{n} \]

\[ s_{xi}^2 = \Sigma_{j=1}^{n_{xi}} \frac{(x_{xj}- \bar x_i)^2}{n_{xi}-1} \]

\[ s_{yi}^2 = \Sigma_{j=1}^{n_{yi}} \frac{(y_{ij}- \bar y_i)^2}{n_{yi}-1} \]

\[ s_{\bar d}^2 = \Sigma_{j=1}^{n} \frac{(d_{i}- \bar d)^2}{n-1} \]

Step 2: Compute pooled estimates of within subject errors

\[ s^2_{xw} = \Sigma_{i=1}^{n} [\frac{n_{xi} -1}{N_x -1} \cdot s^2_{xi}] \]

\[ s^2_{yw} = \Sigma_{i=1}^{n} [\frac{n_{yi} -1}{N_y -1} \cdot s^2_{yi}] \]

Step 3: Compute Harmonic Means of Replicates

\[ m_{xh} = \frac{n}{\Sigma_{i=1}^n \frac{1}{n_{xi}}} \]

\[ m_{yh} = \frac{n}{\Sigma_{i=1}^n \frac{1}{n_{yi}}} \] Step 4: Compute the variance of the differences

\[ s^2_d = s^2_{\bar d} + (1+\frac{1}{m_{xh}}) \cdot s^2_{xw} + (1+\frac{1}{m_{yh}}) \cdot s^2_{yw} \] Step 5: Compute MOVER Components

\[ S_{11} = s_{\bar d}^2 \cdot (1 - \frac{n-1}{\chi^2_{(1-\alpha, n-1)}}) \]

\[ S_{12} = (1-\frac{1}{m_{xh}}) \cdot (1 - \frac{N_x-n}{\chi^2_{(1-\alpha, N_x-n)}}) \cdot s^2_{xw} \] \[ S_{13} = (1-\frac{1}{m_{yh}}) \cdot (1 - \frac{N_y-n}{\chi^2_{(1-\alpha, N_y-n)}}) \cdot s^2_{yw} \]

\[ S_1 = \sqrt{S_{11}^2 +S_{12}^2 +S_{13}^2} \] \[ l = s_d^2 - S_1 \]

\[ u = s_d^2 + S_1 \]

\[ LME = \sqrt{\frac{z^2_{\alpha} \cdot s_d^2}{n} + z^2_{\beta/2} \cdot(\sqrt{u} - \sqrt{s^2_d})^2} \]

\[ RME = \sqrt{\frac{z^2_{\alpha} \cdot s_d^2}{n} + z^2_{\beta/2} \cdot(\sqrt{l} - \sqrt{s^2_d})^2} \] ### LoA

\[ LoA_{lower} = \bar d - z_{\beta/2} \cdot s_d \]

\[ LoA_{upper} = \bar d + z_{\beta/2} \cdot s_d \]

agree_reps LoA

CCC from U-statistics

The CCC calculations are derived from the cccUst function of the cccrm R package. The mathematics for this CCC calculation can be found in the work of King, Chinchilli, and Carrasco (2007) and Carrasco, King, and Chinchilli (2009).

An Example

a1 = agree_test(x = reps$x,
                y = reps$y,
                agree.level = .8)

check(a1)