The kappa statistic implemented by Fleiss is a very popular index for assessing the reliability of agreement among multiple observers. It is used both in the psychological and in the psychiatric field. Other fields of application are typically medicine, biology and engineering. Unfortunately,the kappa statistic may behave inconsistently in case of strong agreement between raters, since this index assumes lower values than it would have been expected. We propose a modification kappa implemented by Fleiss in case of nominal and ordinal variables. Monte Carlo simulations are used both to testing statistical hypotheses and to calculating percentile bootstrap confidence intervals based on proposed statistic in case of nominal and ordinal data.

Computes confidence intervals for the rate (or risk)
    difference ('RD') or rate ratio (or relative risk, 'RR') for 
    binomial proportions or Poisson rates, or for odds ratio 
    ('OR', binomial only). Also confidence intervals for a single 
    binomial or Poisson rate, and intervals for matched pairs. 
    Includes skewness-corrected asymptotic score ('SCAS') methods, 
    which have been developed in Laud (2017) <doi:10.1002/pst.1813>
    from Miettinen & Nurminen (1985) <doi:10.1002/sim.4780040211> and 
    Gart & Nam (1988) <doi:10.2307/2531848>. The same score produces 
    hypothesis tests analogous to the test for binomial RD and RR by 
    Farrington & Manning (1990) <doi:10.1002/sim.4780091208>. 
    The package also includes MOVER methods
    (Method Of Variance Estimates Recovery) for all contrasts, derived 
    from the Newcombe method but using equal-tailed Jeffreys intervals,
    and generalised for Bayesian applications incorporating prior 
    information. So-called 'exact' methods for strictly conservative 
    coverage are approximated using continuity corrections.
    Also includes methods for stratified calculations (e.g. meta-analysis),
    either assuming fixed effects (matching the CMH test) or incorporating 
    stratum heterogeneity.