The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

modelcardr 0.4.0

modelcardr audits predictions from a model that has already run. It does not retrain the model and does not need the fitted model object.

It supports:

• binary classification
• multiclass classification
• regression

Install

Open modelcardr.Rproj in RStudio and choose Build > Install and Restart, or install the built source package:

install.packages("modelcardr.tar.gz", repos = NULL, type = "source")
library(modelcardr)

Binary classification

audit <- audit_predictions(
  truth = actual_class,
  probability = positive_class_probability,
  positive = "yes",
  bootstrap = 500
)
render_audit(audit, "binary-audit.html")

The audit includes classification metrics, bootstrap intervals, confusion matrix, threshold trade-offs, ROC and precision-recall curves, probability calibration, optional subgroup checks, and optional probability drift.

Multiclass classification

The probability matrix must contain one named column per class.

audit <- audit_predictions(
  truth = actual_class,
  probability = class_probability_matrix,
  task = "multiclass_classification",
  bootstrap = 500
)
render_audit(audit, "multiclass-audit.html")

The audit includes overall metrics, macro and weighted F1, minimum class recall, multiclass log loss and Brier score, one-vs-rest AUC, confusion matrix, class-level metrics, top-label calibration, and subgroup comparisons.

Regression

audit <- audit_predictions(
  truth = actual_value,
  estimate = predicted_value,
  task = "regression",
  bootstrap = 500
)
render_audit(audit, "regression-audit.html")

The audit includes MAE, RMSE, MSE, R-squared, bias, MAPE, SMAPE, correlation, bootstrap intervals, actual-versus-predicted diagnostics, residual analysis, and subgroup error comparisons.

Automatic task detection

task = "auto" is the default. Character and factor outcomes are treated as classification. Numeric truth and numeric estimates are treated as regression. For numeric class codes, set task explicitly.

Pass/fail requirements

rules <- audit_requirements(
  min_accuracy = 0.80,
  min_macro_f1 = 0.75,
  max_expected_calibration_error = 0.05
)

audit <- audit_predictions(
  truth = actual_class,
  probability = class_probability_matrix,
  task = "multiclass_classification",
  requirements = rules
)

audit$overall_status

Regression rules include max_rmse, max_mae, min_r_squared, max_absolute_bias, max_mape, and max_smape.

A requirement that does not apply to the selected task is marked INCONCLUSIVE, rather than silently ignored.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.