The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

probcal 0.2.0

• Added a calibration-inference layer for binary and multiclass predictions.
  • skce() computes the squared kernel calibration error of Widmann et al. (2019) with three estimators: the unbiased U-statistic ("uq"), the unbiased linear-time estimator ("ul"), and the biased V-statistic ("biased"). It supports the binary/confidence reduction, the strong (canonical) multiclass form via a matrix-valued kernel, and the classwise one-vs-rest reduction. In the binary and confidence cases mmce() equals sqrt(skce(..., estimator = "biased")).
  • cal_test() performs a kernel calibration hypothesis test (H0: the model is calibrated). The default method = "bootstrap" uses the more powerful quadratic estimator with a wild bootstrap (Widmann et al. 2019, Theorem G.2); method = "asymptotic" uses the faster linear-estimator normal test (Lemma 3). It returns an object of class c("cal_test", "htest"). The test targets are binary, "confidence", and the strong "canonical" multiclass form; the classwise average is available only as a point estimate from skce().
  • cal_ci() returns a percentile bootstrap confidence interval for ece(), skce(), mmce(), mce(), or ace(), as a classed cal_ci object with a print() method.
  • skce() and cal_test() accept bandwidth = "median" for the median-heuristic kernel scale (Widmann et al. 2019), recommended for the canonical multiclass form; the fixed 0.2 remains the default.
• ece() gains debiased (the debiased squared-ECE estimator of Kumar, Liang & Ma 2019, Definition 5.2), strategy (equal-width or equal-mass bins, Roelofs et al. 2022), and norm ("l1" or "l2"). The norm and debiased choices are independent; debiasing is defined only for norm = "l2". Defaults (norm = "l1", debiased = FALSE, strategy = "width") reproduce the previous numeric output exactly.
• stratified_folds() (used by cal_cv()) now scopes the optional fold-assignment seed with withr::local_seed() instead of touching .Random.seed directly; withr was added to Imports. The fold assignments for a given seed are unchanged.
• inst/CITATION now reads the version from the package metadata, and the package author is also declared as copyright holder in Authors@R.
• The README documents the calibration-inference layer (skce(), cal_test(), cal_ci(), and the debiased, strategy, and norm arguments of ece()) with a worked example.

probcal 0.1.0

• CRAN submission version.

probcal 0.0.0.9000

• Added multiclass calibration. cal_temperature() and cal_cv() accept a logit or probability matrix, and new constructors cal_vector_scaling(), cal_dirichlet(), and cal_ovr() cover vector scaling, Dirichlet calibration, and one-vs-rest calibration.
• ece(), mce(), ace(), and reliability_diagram() accept a probability matrix with a type argument for classwise or top-label confidence evaluation.
• Added mmce(), a binning-free Maximum Mean Calibration Error metric for binary and multiclass predictions.
• Added inst/CITATION so users can cite the package with citation("probcal").
• Added applied workflow, calibrator selection, and numerical validation vignettes.
• print() and summary() respect options(probcal.emoji = FALSE) to suppress the decorative glyph in console output.
• reliability_diagram() now reports ECE in the subtitle by default and can use either count-scaled or fixed-size points.
• Added optional development validation tests against Python netcal and R betacal.
• Initial development version with binary calibration methods, calibration metrics, reliability diagrams, and out-of-fold calibration support.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.