The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

hmetad

The hmetad package is designed to fit the meta-d’ model for confidence ratings (Maniscalco & Lau, 2012, 2014). The hmetad package uses a Bayesian modeling approach, building on and superseding previous development of the Hmeta-d toolbox (Fleming, 2017). A key advance is implementation as a custom family in the brms package, which itself provides a friendly interface to the probabilistic programming language Stan.

This provides major benefits:

Model designs can be specified as simple R formulas
Support for complex model designs (e.g., multilevel models, distributional models, multivariate models)
Interfaces to other packages surrounding brms (e.g., tidybayes, ggdist, bayesplot, loo, posterior, bridgesampling, bayestestR)
Computation of model-implied quantities (e.g., mean confidence, type 1 and type 2 receiver operating characteristic curves)
Derivation of novel metacognitive bias metrics, and well as established metrics of metacognitive efficiency
Increased sampling efficiency and better convergence diagnostics

Installation

hmetad is available via CRAN and can be installed using:

install.packages("hmetad")

Alternatively, you can install the development version of hmetad from GitHub with:

# install.packages("pak")
pak::pak("metacoglab/hmetad")

Quick setup

Let’s say you have some data from a binary decision task with ordinal confidence ratings:

#> # A tibble: 1,000 × 5
#>    trial stimulus response correct confidence
#>    <int>    <int>    <int>   <int>      <int>
#>  1     1        0        0       1          2
#>  2     2        1        1       1          4
#>  3     3        1        1       1          2
#>  4     4        0        0       1          4
#>  5     5        0        1       0          1
#>  6     6        1        1       1          2
#>  7     7        0        0       1          2
#>  8     8        1        0       0          1
#>  9     9        1        1       1          1
#> 10    10        0        1       0          3
#> # ℹ 990 more rows

You can fit an intercepts-only meta-d’ model using fit_metad:

library(hmetad)

m <- fit_metad(N ~ 1,
  data = d,
  prior = prior(normal(0, 1), class = Intercept) +
    set_prior(
      "normal(0, 1)",
      class = c(
        "dprime", "c",
        "metac2zero1diff", "metac2zero2diff", "metac2zero3diff",
        "metac2one1diff", "metac2one2diff", "metac2one3diff"
      )
    )
)

#>  Family: metad__4__normal__absolute__multinomial 
#>   Links: mu = log 
#> Formula: N ~ 1 
#>    Data: data.aggregated (Number of observations: 1) 
#>   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup draws = 4000
#> 
#> Regression Coefficients:
#>           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept    -0.04      0.14    -0.35     0.23 1.00     3115     3092
#> 
#> Further Distributional Parameters:
#>                 Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> dprime              1.05      0.08     0.90     1.22 1.00     4058     2832
#> c                  -0.04      0.04    -0.12     0.04 1.00     3950     3312
#> metac2zero1diff     0.47      0.03     0.41     0.54 1.00     5121     3320
#> metac2zero2diff     0.49      0.04     0.42     0.57 1.00     5262     2913
#> metac2zero3diff     0.56      0.05     0.47     0.66 1.00     5578     3139
#> metac2one1diff      0.50      0.04     0.44     0.57 1.00     4059     3253
#> metac2one2diff      0.52      0.04     0.45     0.60 1.00     5753     3401
#> metac2one3diff      0.46      0.04     0.37     0.55 1.00     5955     2555
#> 
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

Now let’s say you have a more complicated design, such as a within-participant manipulation:

#> # A tibble: 5,000 × 7
#> # Groups:   participant, condition [50]
#>    participant condition trial stimulus response correct confidence
#>          <int>     <int> <int>    <int>    <int>   <int>      <int>
#>  1           1         1     1        0        0       1          4
#>  2           1         1     2        1        1       1          4
#>  3           1         1     3        0        0       1          3
#>  4           1         1     4        0        1       0          3
#>  5           1         1     5        0        0       1          3
#>  6           1         1     6        0        0       1          4
#>  7           1         1     7        0        0       1          4
#>  8           1         1     8        0        0       1          1
#>  9           1         1     9        0        1       0          1
#> 10           1         1    10        1        0       0          3
#> # ℹ 4,990 more rows

To account for the repeated measures in this design, you can simply adjust the formula to include participant-level effects:

m <- fit_metad(
  bf(
    N ~ condition + (condition | participant),
    dprime + c +
      metac2zero1diff + metac2zero2diff + metac2zero3diff +
      metac2one1diff + metac2one2diff + metac2one3diff ~
      condition + (condition | participant)
  ),
  data = d, init = "0",
  prior = prior(normal(0, 1)) +
    set_prior("normal(0, 1)",
      dpar = c(
        "dprime", "c",
        "metac2zero1diff", "metac2zero2diff", "metac2zero3diff",
        "metac2one1diff", "metac2one2diff", "metac2one3diff"
      )
    )
)

#>  Family: metad__4__normal__absolute__multinomial 
#>   Links: mu = log; dprime = identity; c = identity; metac2zero1diff = log; metac2zero2diff = log; metac2zero3diff = log; metac2one1diff = log; metac2one2diff = log; metac2one3diff = log 
#> Formula: N ~ condition + (condition | participant) 
#>          dprime ~ condition + (condition | participant)
#>          c ~ condition + (condition | participant)
#>          metac2zero1diff ~ condition + (condition | participant)
#>          metac2zero2diff ~ condition + (condition | participant)
#>          metac2zero3diff ~ condition + (condition | participant)
#>          metac2one1diff ~ condition + (condition | participant)
#>          metac2one2diff ~ condition + (condition | participant)
#>          metac2one3diff ~ condition + (condition | participant)
#>    Data: data.aggregated (Number of observations: 50) 
#>   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#>          total post-warmup draws = 4000
#> 
#> Multilevel Hyperparameters:
#> ~participant (Number of levels: 25) 
#>                                                          Estimate Est.Error
#> sd(Intercept)                                                0.49      0.52
#> sd(condition)                                                0.31      0.32
#> sd(dprime_Intercept)                                         1.22      0.22
#> sd(dprime_condition)                                         0.75      0.14
#> sd(c_Intercept)                                              1.16      0.17
#> sd(c_condition)                                              0.74      0.11
#> sd(metac2zero1diff_Intercept)                                0.11      0.10
#> sd(metac2zero1diff_condition)                                0.07      0.06
#> sd(metac2zero2diff_Intercept)                                0.11      0.11
#> sd(metac2zero2diff_condition)                                0.07      0.07
#> sd(metac2zero3diff_Intercept)                                0.10      0.09
#> sd(metac2zero3diff_condition)                                0.07      0.06
#> sd(metac2one1diff_Intercept)                                 0.10      0.10
#> sd(metac2one1diff_condition)                                 0.07      0.08
#> sd(metac2one2diff_Intercept)                                 0.14      0.11
#> sd(metac2one2diff_condition)                                 0.11      0.08
#> sd(metac2one3diff_Intercept)                                 0.13      0.12
#> sd(metac2one3diff_condition)                                 0.07      0.06
#> cor(Intercept,condition)                                    -0.43      0.57
#> cor(dprime_Intercept,dprime_condition)                      -0.94      0.03
#> cor(c_Intercept,c_condition)                                -0.94      0.03
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition)    -0.41      0.56
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition)    -0.34      0.56
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition)    -0.37      0.56
#> cor(metac2one1diff_Intercept,metac2one1diff_condition)      -0.42      0.54
#> cor(metac2one2diff_Intercept,metac2one2diff_condition)      -0.32      0.55
#> cor(metac2one3diff_Intercept,metac2one3diff_condition)      -0.21      0.67
#>                                                          l-95% CI u-95% CI Rhat
#> sd(Intercept)                                                0.02     2.02 1.18
#> sd(condition)                                                0.01     1.26 1.19
#> sd(dprime_Intercept)                                         0.89     1.74 1.02
#> sd(dprime_condition)                                         0.56     1.07 1.06
#> sd(c_Intercept)                                              0.88     1.52 1.04
#> sd(c_condition)                                              0.55     0.97 1.04
#> sd(metac2zero1diff_Intercept)                                0.00     0.35 1.05
#> sd(metac2zero1diff_condition)                                0.00     0.24 1.02
#> sd(metac2zero2diff_Intercept)                                0.00     0.42 1.10
#> sd(metac2zero2diff_condition)                                0.00     0.27 1.05
#> sd(metac2zero3diff_Intercept)                                0.01     0.35 1.02
#> sd(metac2zero3diff_condition)                                0.00     0.24 1.02
#> sd(metac2one1diff_Intercept)                                 0.00     0.41 1.01
#> sd(metac2one1diff_condition)                                 0.00     0.29 1.04
#> sd(metac2one2diff_Intercept)                                 0.01     0.41 1.07
#> sd(metac2one2diff_condition)                                 0.01     0.30 1.04
#> sd(metac2one3diff_Intercept)                                 0.00     0.45 1.02
#> sd(metac2one3diff_condition)                                 0.00     0.24 1.03
#> cor(Intercept,condition)                                    -1.00     0.88 1.17
#> cor(dprime_Intercept,dprime_condition)                      -0.98    -0.87 1.02
#> cor(c_Intercept,c_condition)                                -0.98    -0.88 1.11
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition)    -0.99     0.86 1.03
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition)    -0.99     0.88 1.03
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition)    -0.99     0.87 1.02
#> cor(metac2one1diff_Intercept,metac2one1diff_condition)      -0.99     0.85 1.03
#> cor(metac2one2diff_Intercept,metac2one2diff_condition)      -0.98     0.85 1.09
#> cor(metac2one3diff_Intercept,metac2one3diff_condition)      -0.99     0.95 1.13
#>                                                          Bulk_ESS Tail_ESS
#> sd(Intercept)                                                  16       11
#> sd(condition)                                                  15       13
#> sd(dprime_Intercept)                                          401     1149
#> sd(dprime_condition)                                           56      591
#> sd(c_Intercept)                                               115      689
#> sd(c_condition)                                               146      575
#> sd(metac2zero1diff_Intercept)                                  56     1006
#> sd(metac2zero1diff_condition)                                 235      896
#> sd(metac2zero2diff_Intercept)                                  28      283
#> sd(metac2zero2diff_condition)                                 769     1288
#> sd(metac2zero3diff_Intercept)                                 510     1522
#> sd(metac2zero3diff_condition)                                 728     1099
#> sd(metac2one1diff_Intercept)                                  478     1206
#> sd(metac2one1diff_condition)                                  128      550
#> sd(metac2one2diff_Intercept)                                 1167     1470
#> sd(metac2one2diff_condition)                                  465      791
#> sd(metac2one3diff_Intercept)                                  535     1083
#> sd(metac2one3diff_condition)                                  599     1076
#> cor(Intercept,condition)                                       17       30
#> cor(dprime_Intercept,dprime_condition)                        395     1404
#> cor(c_Intercept,c_condition)                                   26       73
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition)      119     1441
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition)      143     1646
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition)      828     1474
#> cor(metac2one1diff_Intercept,metac2one1diff_condition)        657     1283
#> cor(metac2one2diff_Intercept,metac2one2diff_condition)        607     1508
#> cor(metac2one3diff_Intercept,metac2one3diff_condition)         22       77
#> 
#> Regression Coefficients:
#>                           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
#> Intercept                     0.48      0.31    -0.14     1.02 1.14       19
#> dprime_Intercept              0.33      0.26    -0.19     0.86 1.06      398
#> c_Intercept                  -0.10      0.23    -0.56     0.35 1.03      230
#> metac2zero1diff_Intercept    -1.02      0.12    -1.27    -0.79 1.02      243
#> metac2zero2diff_Intercept    -1.04      0.14    -1.30    -0.77 1.02      219
#> metac2zero3diff_Intercept    -1.09      0.15    -1.38    -0.82 1.03      126
#> metac2one1diff_Intercept     -1.07      0.13    -1.33    -0.84 1.02      167
#> metac2one2diff_Intercept     -0.90      0.13    -1.16    -0.65 1.05     2504
#> metac2one3diff_Intercept     -0.95      0.14    -1.21    -0.68 1.01      343
#> condition                    -0.27      0.17    -0.60     0.03 1.11       24
#> dprime_condition              0.38      0.17     0.06     0.71 1.03      175
#> c_condition                   0.10      0.15    -0.19     0.41 1.04      194
#> metac2zero1diff_condition     0.06      0.08    -0.09     0.21 1.02      179
#> metac2zero2diff_condition     0.01      0.09    -0.17     0.17 1.04       82
#> metac2zero3diff_condition     0.01      0.09    -0.17     0.19 1.01      945
#> metac2one1diff_condition      0.01      0.08    -0.15     0.17 1.01      623
#> metac2one2diff_condition     -0.09      0.08    -0.26     0.07 1.05     2112
#> metac2one3diff_condition     -0.03      0.09    -0.20     0.14 1.04     1436
#>                           Tail_ESS
#> Intercept                       19
#> dprime_Intercept               734
#> c_Intercept                    601
#> metac2zero1diff_Intercept     1465
#> metac2zero2diff_Intercept     2002
#> metac2zero3diff_Intercept     2022
#> metac2one1diff_Intercept      1872
#> metac2one2diff_Intercept      2062
#> metac2one3diff_Intercept      2198
#> condition                       46
#> dprime_condition               682
#> c_condition                    604
#> metac2zero1diff_condition     1865
#> metac2zero2diff_condition     1828
#> metac2zero3diff_condition     1569
#> metac2one1diff_condition      1991
#> metac2one2diff_condition      2074
#> metac2one3diff_condition      1557
#> 
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).

References

Fleming, S. M. (2017). HMeta-d: Hierarchical bayesian estimation of metacognitive efficiency from confidence ratings. Neuroscience of Consciousness, 2017(1), nix007.

Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition, 21(1), 422–430.

Maniscalco, B., & Lau, H. (2014). Signal detection theory analysis of type 1 and type 2 data: Meta-d′, response-specific meta-d′, and the unequal variance SDT model. In The cognitive neuroscience of metacognition (pp. 25–66). Springer.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.