The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

hmetad

The hmetad package is designed to fit the meta-d’ model for confidence ratings (Maniscalco & Lau, 2012, 2014). The hmetad package uses a Bayesian modeling approach, building on and superseding previous development of the Hmeta-d toolbox (Fleming, 2017). A key advance is implementation as a custom family in the brms package, which itself provides a friendly interface to the probabilistic programming language Stan.

This provides major benefits:

Model designs can be specified as simple R formulas
Support for complex model designs (e.g., multilevel models, distributional models, multivariate models)
Interfaces to other packages surrounding brms (e.g., tidybayes, ggdist, bayesplot, loo, posterior, bridgesampling, bayestestR)
Computation of model-implied quantities (e.g., mean confidence, type 1 and type 2 receiver operating characteristic curves)
Derivation of novel metacognitive bias metrics, and well as established metrics of metacognitive efficiency
Increased sampling efficiency and better convergence diagnostics

Installation

hmetad is available via CRAN and can be installed using:

install.packages("hmetad")

Alternatively, you can install the development version of hmetad from GitHub with:

# install.packages("pak")
pak::pak("metacoglab/hmetad")

Quick setup

Let’s say you have some data from a binary decision task with ordinal confidence ratings:

#> # A tibble: 1,000 × 5
#>    trial stimulus response correct confidence
#>    <int>    <int>    <int>   <int>      <int>
#>  1     1        1        0       0          1
#>  2     2        0        0       1          2
#>  3     3        1        1       1          3
#>  4     4        0        1       0          2
#>  5     5        0        0       1          3
#>  6     6        0        0       1          3
#>  7     7        0        0       1          3
#>  8     8        0        1       0          3
#>  9     9        1        0       0          2
#> 10    10        0        1       0          3
#> # ℹ 990 more rows

You can fit an intercepts-only meta-d’ model using fit_metad:

library(hmetad)

m <- fit_metad(N ~ 1,
  data = d,
  prior = prior(normal(0, 1), class = Intercept) +
    set_prior("normal(0, 1)", class = c("dprime", "c", metac2_parameters(K = 4)))
)

#>  Family: metad__4__normal__absolute__multinomial 
#>   Links: mu = log 
#> Formula: N ~ 1 
#>    Data: data.aggregated (Number of observations: 1) 
#>   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup draws = 4000
#> 
#> Regression Coefficients:
#>           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept     0.09      0.15    -0.21     0.37 1.00     3361     2972
#> 
#> Further Distributional Parameters:
#>                 Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> dprime              0.96      0.08     0.81     1.13 1.00     4198     3420
#> c                  -0.03      0.04    -0.11     0.05 1.00     3956     2995
#> metac2zero1diff     0.52      0.04     0.45     0.59 1.00     4969     2878
#> metac2zero2diff     0.45      0.04     0.38     0.53 1.00     4413     3006
#> metac2zero3diff     0.55      0.05     0.45     0.65 1.00     5463     2815
#> metac2one1diff      0.55      0.04     0.48     0.62 1.00     5035     3158
#> metac2one2diff      0.53      0.04     0.45     0.62 1.00     5315     3096
#> metac2one3diff      0.55      0.05     0.45     0.65 1.00     5463     2880
#> 
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

Now let’s say you have a more complicated design, such as a within-participant manipulation:

#> # A tibble: 5,000 × 7
#> # Groups:   participant, condition [50]
#>    participant condition trial stimulus response correct confidence
#>          <int>     <int> <int>    <int>    <int>   <int>      <int>
#>  1           1         1     1        1        1       1          4
#>  2           1         1     2        1        1       1          1
#>  3           1         1     3        1        1       1          2
#>  4           1         1     4        0        0       1          2
#>  5           1         1     5        0        1       0          1
#>  6           1         1     6        0        0       1          4
#>  7           1         1     7        0        0       1          4
#>  8           1         1     8        1        1       1          2
#>  9           1         1     9        0        0       1          1
#> 10           1         1    10        0        0       1          4
#> # ℹ 4,990 more rows

To account for the repeated measures in this design, you can simply adjust the formula to include participant-level effects:

m <- fit_metad(
  bf(
    N ~ condition + (condition | participant),
    dprime + c +
      metac2zero1diff + metac2zero2diff + metac2zero3diff +
      metac2one1diff + metac2one2diff + metac2one3diff ~
      condition + (condition | participant)
  ),
  data = d, init = "0",
  prior = prior(normal(0, 1)) +
    set_prior("normal(0, 1)", dpar = c("dprime", "c", metac2_parameters(K = 4)))
)

#>  Family: metad__4__normal__absolute__multinomial 
#>   Links: mu = log; dprime = identity; c = identity; metac2zero1diff = log; metac2zero2diff = log; metac2zero3diff = log; metac2one1diff = log; metac2one2diff = log; metac2one3diff = log 
#> Formula: N ~ condition + (condition | participant) 
#>          dprime ~ condition + (condition | participant)
#>          c ~ condition + (condition | participant)
#>          metac2zero1diff ~ condition + (condition | participant)
#>          metac2zero2diff ~ condition + (condition | participant)
#>          metac2zero3diff ~ condition + (condition | participant)
#>          metac2one1diff ~ condition + (condition | participant)
#>          metac2one2diff ~ condition + (condition | participant)
#>          metac2one3diff ~ condition + (condition | participant)
#>    Data: data.aggregated (Number of observations: 50) 
#>   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup draws = 4000
#> 
#> Multilevel Hyperparameters:
#> ~participant (Number of levels: 25) 
#>                                                          Estimate Est.Error
#> sd(Intercept)                                                0.37      0.23
#> sd(condition)                                                0.21      0.15
#> sd(dprime_Intercept)                                         1.35      0.23
#> sd(dprime_condition)                                         0.89      0.15
#> sd(c_Intercept)                                              1.30      0.20
#> sd(c_condition)                                              0.89      0.14
#> sd(metac2zero1diff_Intercept)                                0.20      0.19
#> sd(metac2zero1diff_condition)                                0.15      0.13
#> sd(metac2zero2diff_Intercept)                                0.08      0.07
#> sd(metac2zero2diff_condition)                                0.05      0.04
#> sd(metac2zero3diff_Intercept)                                0.08      0.07
#> sd(metac2zero3diff_condition)                                0.05      0.04
#> sd(metac2one1diff_Intercept)                                 0.17      0.18
#> sd(metac2one1diff_condition)                                 0.09      0.10
#> sd(metac2one2diff_Intercept)                                 0.18      0.18
#> sd(metac2one2diff_condition)                                 0.12      0.12
#> sd(metac2one3diff_Intercept)                                 0.12      0.10
#> sd(metac2one3diff_condition)                                 0.08      0.06
#> cor(Intercept,condition)                                    -0.35      0.55
#> cor(dprime_Intercept,dprime_condition)                      -0.96      0.02
#> cor(c_Intercept,c_condition)                                -0.96      0.02
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition)    -0.57      0.54
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition)    -0.29      0.58
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition)    -0.32      0.57
#> cor(metac2one1diff_Intercept,metac2one1diff_condition)      -0.47      0.57
#> cor(metac2one2diff_Intercept,metac2one2diff_condition)      -0.50      0.56
#> cor(metac2one3diff_Intercept,metac2one3diff_condition)      -0.33      0.56
#>                                                          l-95% CI u-95% CI Rhat
#> sd(Intercept)                                                0.03     0.92 1.00
#> sd(condition)                                                0.01     0.60 1.01
#> sd(dprime_Intercept)                                         0.97     1.86 1.01
#> sd(dprime_condition)                                         0.64     1.22 1.00
#> sd(c_Intercept)                                              0.98     1.77 1.01
#> sd(c_condition)                                              0.67     1.24 1.01
#> sd(metac2zero1diff_Intercept)                                0.00     0.64 1.01
#> sd(metac2zero1diff_condition)                                0.00     0.46 1.01
#> sd(metac2zero2diff_Intercept)                                0.00     0.26 1.00
#> sd(metac2zero2diff_condition)                                0.00     0.16 1.00
#> sd(metac2zero3diff_Intercept)                                0.00     0.26 1.00
#> sd(metac2zero3diff_condition)                                0.00     0.16 1.00
#> sd(metac2one1diff_Intercept)                                 0.00     0.66 1.00
#> sd(metac2one1diff_condition)                                 0.00     0.37 1.00
#> sd(metac2one2diff_Intercept)                                 0.01     0.64 1.01
#> sd(metac2one2diff_condition)                                 0.00     0.43 1.01
#> sd(metac2one3diff_Intercept)                                 0.01     0.37 1.00
#> sd(metac2one3diff_condition)                                 0.00     0.23 1.00
#> cor(Intercept,condition)                                    -0.97     0.88 1.01
#> cor(dprime_Intercept,dprime_condition)                      -0.99    -0.91 1.01
#> cor(c_Intercept,c_condition)                                -0.98    -0.91 1.01
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition)    -1.00     0.81 1.01
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition)    -0.99     0.91 1.00
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition)    -0.99     0.88 1.00
#> cor(metac2one1diff_Intercept,metac2one1diff_condition)      -1.00     0.84 1.00
#> cor(metac2one2diff_Intercept,metac2one2diff_condition)      -1.00     0.85 1.00
#> cor(metac2one3diff_Intercept,metac2one3diff_condition)      -0.98     0.89 1.00
#>                                                          Bulk_ESS Tail_ESS
#> sd(Intercept)                                                1038     1395
#> sd(condition)                                                 497      615
#> sd(dprime_Intercept)                                         1237     2085
#> sd(dprime_condition)                                         1198     2025
#> sd(c_Intercept)                                               474      677
#> sd(c_condition)                                               430      387
#> sd(metac2zero1diff_Intercept)                                 455     1503
#> sd(metac2zero1diff_condition)                                 409     1210
#> sd(metac2zero2diff_Intercept)                                1982     2046
#> sd(metac2zero2diff_condition)                                1934     1606
#> sd(metac2zero3diff_Intercept)                                2139     2019
#> sd(metac2zero3diff_condition)                                1270      706
#> sd(metac2one1diff_Intercept)                                  677     1368
#> sd(metac2one1diff_condition)                                  824     1303
#> sd(metac2one2diff_Intercept)                                  688     1714
#> sd(metac2one2diff_condition)                                  575     1858
#> sd(metac2one3diff_Intercept)                                 1779     2015
#> sd(metac2one3diff_condition)                                 1124     1708
#> cor(Intercept,condition)                                     1032     1778
#> cor(dprime_Intercept,dprime_condition)                       1125     1767
#> cor(c_Intercept,c_condition)                                  616     1643
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition)      570     2153
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition)     2919     2624
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition)     2425     2209
#> cor(metac2one1diff_Intercept,metac2one1diff_condition)       1071     2089
#> cor(metac2one2diff_Intercept,metac2one2diff_condition)        983     2289
#> cor(metac2one3diff_Intercept,metac2one3diff_condition)       1866     2267
#> 
#> Regression Coefficients:
#>                           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
#> Intercept                    -0.34      0.24    -0.83     0.13 1.00     1465
#> dprime_Intercept              1.21      0.30     0.60     1.77 1.01     1131
#> c_Intercept                   0.12      0.26    -0.38     0.64 1.01      417
#> metac2zero1diff_Intercept    -0.82      0.13    -1.09    -0.57 1.00     4220
#> metac2zero2diff_Intercept    -0.88      0.13    -1.13    -0.62 1.00     5486
#> metac2zero3diff_Intercept    -1.18      0.14    -1.47    -0.90 1.00     3378
#> metac2one1diff_Intercept     -1.19      0.14    -1.48    -0.92 1.00     3432
#> metac2one2diff_Intercept     -0.91      0.14    -1.20    -0.63 1.00     2572
#> metac2one3diff_Intercept     -1.26      0.15    -1.57    -0.96 1.00     4428
#> condition                     0.25      0.16    -0.08     0.54 1.00     1387
#> dprime_condition             -0.14      0.19    -0.51     0.25 1.01     1125
#> c_condition                  -0.02      0.17    -0.37     0.31 1.01      442
#> metac2zero1diff_condition    -0.11      0.09    -0.28     0.06 1.00     4038
#> metac2zero2diff_condition    -0.13      0.08    -0.30     0.03 1.00     5083
#> metac2zero3diff_condition     0.11      0.09    -0.05     0.29 1.00     4423
#> metac2one1diff_condition      0.10      0.09    -0.07     0.27 1.00     3650
#> metac2one2diff_condition     -0.06      0.09    -0.24     0.13 1.00     2197
#> metac2one3diff_condition      0.17      0.10    -0.02     0.37 1.00     4381
#>                           Tail_ESS
#> Intercept                      623
#> dprime_Intercept              1845
#> c_Intercept                    556
#> metac2zero1diff_Intercept     2332
#> metac2zero2diff_Intercept     2687
#> metac2zero3diff_Intercept     2286
#> metac2one1diff_Intercept      1896
#> metac2one2diff_Intercept      1228
#> metac2one3diff_Intercept      2409
#> condition                      526
#> dprime_condition              1771
#> c_condition                    593
#> metac2zero1diff_condition     2132
#> metac2zero2diff_condition     2759
#> metac2zero3diff_condition     2967
#> metac2one1diff_condition      2255
#> metac2one2diff_condition      1370
#> metac2one3diff_condition      2362
#> 
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

References

Fleming, S. M. (2017). HMeta-d: Hierarchical bayesian estimation of metacognitive efficiency from confidence ratings. Neuroscience of Consciousness, 2017(1), nix007.

Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition, 21(1), 422–430.

Maniscalco, B., & Lau, H. (2014). Signal detection theory analysis of type 1 and type 2 data: Meta-d′, response-specific meta-d′, and the unequal variance SDT model. In The cognitive neuroscience of metacognition (pp. 25–66). Springer.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.