The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

Distributional Regression K > 1 and Residual Diagnostics with DHARMa

Tutorial for fitting per-slot AMM with custom families and validating fits via G1 / G2 / G3 residuals (Path 1)

José Mauricio Gómez Julián

2026-07-06

1. What this vignette covers

gdpar supports per-slot AMM canonical decomposition for distributional regression — that is, fitting one AMM per parameter of a distribution (K > 1). Sub-phases 8.3.4 through 8.3.7 brought online the following library of K > 1 likelihoods:

Bi-parametric (K = 2): Beta, Gamma, log-normal (location-scale), Gaussian.
Tri-parametric (K = 3): Student-t (location, scale, degrees of freedom), Tweedie (location, dispersion, power index in $(1.01, 1.99)$).
Mixtures: zero-inflated Poisson (ZIP, K = 2), zero-inflated negative binomial (ZINB, K = 3), Hurdle-Poisson (K = 2), Hurdle-negative-binomial (K = 3).
Heterogeneous families per slot (K = 2 combinations of compatible families; see vignette("vop04_amm_intermediate", package = "gdpar")).

This vignette covers two complementary topics:

The API for declaring and fitting a K > 1 model.
The residual / posterior-predictive workflow that complements the fit, including the optional integration with the DHARMa package.

For the intermediate AMM specifications (B-spline W bases, heterogeneous families per slot), see vignette("vop04_amm_intermediate", package = "gdpar").

2. The `K > 1` API

2.1. Three equivalent input forms

gdpar() accepts three syntactically equivalent ways of declaring a K > 1 distributional regression. All three canonicalise to the same internal gdpar_formula_set object (sub-phase 8.3.3, decision E):

library(gdpar)

# (a) brms-style `bf()` sugar
fit <- gdpar(
  gdpar_bf(y ~ a(x1), sigma ~ a(x2)),
  data = d, family = gdpar_family("gaussian")
)

# (b) Named list of formulas
fit <- gdpar(
  list(mu = y ~ a(x1), sigma = ~ a(x2)),
  data = d, family = gdpar_family("gaussian")
)

# (c) Named list of amm_spec (low-level, bypasses formula parsing)
fit <- gdpar(
  list(
    mu    = amm_spec(a = ~ x1),
    sigma = amm_spec(a = ~ x2)
  ),
  data = d, family = gdpar_family("gaussian")
)

Three contract notes:

The slot names of the gdpar_formula_set must match the eligible param_specs of the family. For gaussian, the eligibles are {mu, log_sigma}; the alias sigma is canonicalised to log_sigma at construction time. For other families see gdpar_family(name)$param_specs.
The left-hand-side response y appears only in the first formula (the observation slot, slot 1). Subsequent slots use one-sided formulas (~ a(x2)).
Suppressing the anchor via - 1 or + 0 is an error: gdpar canonicalises theta_ref as a structural anchor, not as an optional parameter (sub-phase 8.3.3, decision 5).

2.2. Choosing `K`

The number of slots K is determined by the input. K = 1 retains the legacy path; K = 2 adds dispersion / scale modelling; K = 3 adds shape / weight modelling. The minimum K per family is enforced by .gdpar_guard_K_below_family_min:

Family	`min_K`
`gaussian`, `poisson`, `bernoulli`, `neg_binomial_2`	1
`beta`, `gamma`, `lognormal_loc_scale`	2
`student_t`, `tweedie`	3
`zip`, `hurdle_poisson`	2
`zinb`, `hurdle_neg_binomial_2`	3

A K = 1 fit on a beta family aborts with gdpar_input_error pointing to elevation to K = 2.

The pattern name lognormal_loc_scale is not part of the enum of gdpar_family(name): the package registers it as a K = 2 custom-family pattern (canonised in Sub-phase 8.3.4), accessed via gdpar_family_custom_K(stan_lpdf_id = "lognormal_loc_scale", ...). See §2.4 below for the literal recipe.

2.3. End-to-end example: Gaussian `K = 2`

set.seed(2026L)
n <- 100L
x1 <- rnorm(n); x2 <- rnorm(n)
mu_true       <- 0.4 + 0.6 * (x1 - mean(x1))
log_sigma_eta <- -0.2 + 0.4 * (x2 - mean(x2))
y <- rnorm(n, mu_true, exp(log_sigma_eta))
d <- data.frame(y = y, x1 = x1, x2 = x2)

library(gdpar)
fit_K2 <- gdpar(
  gdpar_bf(y ~ a(x1), sigma ~ a(x2)),
  data   = d,
  family = gdpar_family("gaussian"),
  chains = 2L, iter_warmup = 400L, iter_sampling = 400L,
  refresh = 0L
)

co <- coef(fit_K2)
co$mu
co$sigma

coef.gdpar_fit for K > 1 returns a named list of gdpar_coef objects (decision E4.A, sub-phase 8.3.10). Each entry follows the scalar gdpar_coef contract: posterior summaries of theta_ref, the additive a, the multiplicative b/c_b, and the modulating W. The modulating block is globally shared across slots (replicated identically in every slot’s W component).

2.4. Custom `K > 1` families via `gdpar_family_custom_K()`

The constructor gdpar_family_custom_K() exposes the K-family custom-pattern registry opened in Sub-phase 8.3.4 of Block 8. Each registered pattern is identified by a stan_lpdf_id (the name of a pre-validated Stan _lpdf function shipped with the package) and carries its own minimum K. The first pattern registered is lognormal_loc_scale (min_K = 2); subsequent sub-phases extend the whitelist.

Signature:

gdpar_family_custom_K(
  name,                 # character scalar; must not collide with a built-in
  stan_lpdf_id,         # character scalar; key in the registry
  did_holds     = TRUE, # logical; user declaration of D-ID
  did_condition = NULL, # character scalar describing any conditional D-ID
  did_reference = NULL  # citation supporting did_holds
)

Literal recipe for lognormal_loc_scale (a K = 2 location-scale family on the log scale; slot 1 carries the location and slot 2 carries the log-scale):

my_lognorm <- gdpar_family_custom_K(
  name          = "my_lognormal_K2",
  stan_lpdf_id  = "lognormal_loc_scale",
  did_holds     = TRUE,
  did_reference = "User declaration"
)

fit_lognorm <- gdpar(
  gdpar_bf(y ~ a(x1), sigma ~ a(x2)),
  data   = d,
  family = my_lognorm,
  chains = 2L, iter_warmup = 400L, iter_sampling = 400L,
  refresh = 0L
)

The user is responsible for asserting that the chosen pattern is identifiable in its parameter (did_holds = TRUE); the package does not test identifiability from data, only registers the declaration. Attempting to use an unregistered stan_lpdf_id aborts with a gdpar_input_error that enumerates the allowed patterns. The general (K = 1) custom-family constructor gdpar_family_custom() is documented in §6 below.

2.5. Prediction

# In-sample prediction (theta_i_k draws)
pred_in <- predict(fit_K2, summary = "mean_se")
str(pred_in, max.level = 1L)

# Out-of-sample prediction on new covariates
new_d <- data.frame(x1 = c(-1, 0, 1), x2 = c(-1, 0, 1))
pred_new <- predict(fit_K2, newdata = new_d, summary = "mean_se")
str(pred_new, max.level = 1L)

Three points of contract:

summary = "draws" returns the posterior array of shape $(S, n, K)$, with the third dimension named after the slots.
summary = "mean_se" and summary = "quantiles" return a named list of length K with one summary data frame per slot.
type = "response" applies the slot’s canonical inverse link from object$family$param_specs[[k]]$inv_link. Slot 1 uses the location’s primary link; slot $k > 1$ uses its own canonical link (log for sigma / phi, logit for pi, etc.). Applying the location link to all slots indiscriminately would be incorrect, so the implementation iterates the per-slot link.
newdata on K > 1 polynomial W fits is supported (sub-phase 8.3.10, decision Ruta B + newdata extension). For B-spline W on K > 1 and for grouping (J_groups > 1), the function raises gdpar_unsupported_feature_error and points to Session 8.4 deudas.

3. Residual diagnostics: G1 / G2 / G3

gdpar provides three complementary layers of residual diagnostics (sub-phase 8.3.9, decision D4 ranqueada por máxima robustez):

G1 — deviance and Pearson residuals. Canonical frequentist diagnostics, layer 2 of [feedback-proof-rigor-standards]. Catch bias and heteroscedasticity. Available per-family closed-form where possible; Pearson-like fallback for mixtures and Hurdle.
G2 — randomized quantile residuals (Dunn-Smyth 1996). Bayesian-flavoured layer 1 distributional check. Construct the empirical CDF of the posterior-predictive draws at each observation, evaluate the observation’s quantile, jitter for discrete responses, and map through qnorm() so that residuals are standard normal under the correct model. Sensitive to misspecification of the whole distribution, not just the first two moments.
G3 — posterior predictive checks (PPC). Bayesian layer 3 check. Compare summaries of y_obs to summaries of y_pred draws via bayesplot::pp_check.gdpar_fit. Catches systematic skewness, multimodality, and tail behaviour.

3.1. API

# G1: deviance and Pearson (frequentist canonical)
r_dev  <- residuals(fit_K2, type = "deviance")
r_pear <- residuals(fit_K2, type = "pearson")

# G2: Bayesian quantile residuals (Dunn-Smyth)
r_q <- residuals(fit_K2, type = "quantile", randomize_seed = 1L)

# Response residuals (y_obs - mean of y_pred draws)
r_resp <- residuals(fit_K2, type = "response")

head(data.frame(deviance = r_dev, pearson = r_pear,
                quantile = r_q, response = r_resp))

The signature residuals.gdpar_fit(object, type, coord = NULL, randomize_seed = NULL, ...) lets the user pin the randomisation seed for reproducible G2 residuals across runs. For multi-coordinate fits (p > 1), coord selects which coordinate is summarised.

3.2. Posterior predictive draws and PPC

# Posterior-predictive draws (S x n matrix for K=1 or K>1 with p=1)
pp <- gdpar_posterior_predict(fit_K2)
dim(pp)

# Visual PPCs via bayesplot::pp_check generic
if (requireNamespace("bayesplot", quietly = TRUE)) {
  pp_check(fit_K2, type = "dens_overlay", ndraws = 30L)
}

gdpar_posterior_predict is the exported posterior-predictive draws extractor; pp_check.gdpar_fit is an S3 method off the bayesplot::pp_check generic and supports five PPC types: dens_overlay, hist, ecdf_overlay, stat, intervals. Loading bayesplot makes pp_check(fit_K2) work directly; without it the user can still call pp_check.gdpar_fit(fit_K2) if bayesplot is installed.

4. DHARMa integration (optional)

DHARMa (Hartig 2024) is a popular R package for residual diagnostics that simulates from the fitted model and constructs scaled residuals on $[0, 1]$ for diagnostic plots and formal tests (uniformity, dispersion, outliers, zero-inflation). gdpar integrates with DHARMa via the gdpar_dharma_object() exported function, which constructs a DHARMa simulationOutput from a gdpar_fit. Two points of contract:

DHARMa is a Suggests dependency, not an Imports (sub-phase 8.3.9, decision E1.A). The package’s minimalist Imports (currently {posterior, stats, methods}) is preserved; DHARMa adds ~10 transitive dependencies (lme4, MASS, qrng) that the user opts into.
Detect-and-use pattern. gdpar_dharma_object() checks requireNamespace("DHARMa", quietly = TRUE) and aborts informatively if DHARMa is not installed. Without DHARMa the user can still construct Bayesian quantile residuals via residuals(fit, type = "quantile") — the same Dunn-Smyth methodology underlies both paths.

4.1. API

dh <- gdpar_dharma_object(fit_K2)
class(dh)
#> [1] "DHARMa"
DHARMa::testResiduals(dh)

#> $uniformity
#> 
#>  Asymptotic one-sample Kolmogorov-Smirnov test
#> 
#> data:  simulationOutput$scaledResiduals
#> D = 0.085, p-value = 0.7789
#> alternative hypothesis: two-sided
#> 
#> 
#> $dispersion
#> 
#>  DHARMa nonparametric dispersion test via sd of residuals fitted vs.
#>  simulated
#> 
#> data:  simulationOutput
#> dispersion = 1.0076, p-value = 0.86
#> alternative hypothesis: two.sided
#> 
#> 
#> $outliers
#> 
#>  DHARMa outlier test based on exact binomial test with approximate
#>  expectations
#> 
#> data:  simulationOutput
#> outliers at both margin(s) = 0, observations = 60, p-value = 1
#> alternative hypothesis: true probability of success is not equal to 0.004987531
#> 95 percent confidence interval:
#>  0.00000000 0.05962949
#> sample estimates:
#> frequency of outliers (expected: 0.00498753117206983 ) 
#>                                                      0

The returned object is a standard DHARMa::createDHARMa() simulationOutput with:

nSim: number of posterior-predictive draws used for the empirical CDF (default: all available).
simulatedResponse: matrix of posterior-predictive draws, shape $(n, \text{nSim})$.
observedResponse: the original y_obs.
fittedPredictedResponse: posterior mean of y_pred per observation.

All DHARMa post-processing functions (testUniformity, testDispersion, testOutliers, testZeroInflation, plotResiduals, plotQQunif) work off this object.

4.2. When to use DHARMa vs the built-in G2

The two paths agree on methodology (Bayesian randomized quantile residuals à la Dunn-Smyth 1996). They differ in scope:

Built-in residuals(fit, type = "quantile") returns the residuals as a numeric vector mapped through qnorm() so they are approximately standard normal under the correct model. Useful as input to qqnorm(), shapiro.test(), or any downstream tool that expects standard-normal residuals.
gdpar_dharma_object(fit) returns the residuals as a DHARMa simulationOutput in the $[0, 1]$ scale. Useful when the user wants DHARMa’s plotting machinery and the formal tests for over- / under-dispersion, outliers, and zero-inflation.

Both paths are reproducible: pass randomize_seed to residuals() or set set.seed() before gdpar_dharma_object().

5. Worked example: zero-inflated negative binomial (`K = 3`)

This example exercises both the mixture-likelihood path of sub-phase 8.3.6 and the residual / DHARMa workflow on a tri-parametric K = 3 family.

set.seed(515L)
n <- 120L
x1 <- rnorm(n); x2 <- rnorm(n); x3 <- rnorm(n)
mu_eta    <- 1.0 + 0.5 * (x1 - mean(x1))
log_phi   <- -0.3 + 0.2 * (x2 - mean(x2))
logit_pi  <- -1.0 + 0.6 * (x3 - mean(x3))
mu_true   <- exp(mu_eta)
phi_true  <- exp(log_phi)
pi_true   <- 1 / (1 + exp(-logit_pi))
zero_struc <- rbinom(n, 1, pi_true)
y_count    <- rnbinom(n, size = phi_true, mu = mu_true)
y <- ifelse(zero_struc == 1L, 0L, y_count)
d <- data.frame(y = y, x1 = x1, x2 = x2, x3 = x3)

fit_zinb <- gdpar(
  gdpar_bf(y ~ a(x1), phi ~ a(x2), pi ~ a(x3)),
  data   = d,
  family = gdpar_family("zinb"),
  chains = 2L, iter_warmup = 600L, iter_sampling = 600L,
  refresh = 0L
)

# Per-slot coefficient summary
co <- coef(fit_zinb)
names(co)
co$mu
co$pi

# G2 quantile residuals — robust to mixture structure when jittering
# discrete responses is enabled (default for ZIP/ZINB/hurdle).
r_q <- residuals(fit_zinb, type = "quantile", randomize_seed = 99L)
hist(r_q, breaks = 20L,
     main = "Bayesian quantile residuals — ZINB K=3",
     xlab = "residual")

# DHARMa-side diagnostics if available
if (requireNamespace("DHARMa", quietly = TRUE)) {
  dh <- gdpar_dharma_object(fit_zinb)
  DHARMa::testZeroInflation(dh)
}

For ZIP / ZINB / Hurdle families, gdpar documents the parametrization of pi (zero-inflation / hurdle probability) in the logit scale and the default vectorised prior normal(0, 2.5) per the canonical decision D6 of sub-phase 8.3.6. The pi slot’s coef() output reports the per-term posterior of the AMM acting on logit_pi.

6. Custom family registry: `gdpar_family_custom()` (`K = 1`)

The complement to gdpar_family_custom_K() of §2.4 is the K = 1 constructor gdpar_family_custom(): it builds a fully user-defined family for the legacy single-slot path, where the user supplies the Stan likelihood, the log_lik block (consumed by gdpar_loo()), and the posterior-predictive block (consumed by PPC utilities). Unlike the K-side custom registry, the K = 1 constructor does not draw from a curated whitelist of patterns: any mathematically valid likelihood can be passed verbatim, and the user assumes responsibility both for correctness of the Stan code and for the declaration of identifiability.

Signature:

gdpar_family_custom(
  name,                 # character scalar; must not collide with a built-in
  link,                 # one of "identity", "log", "logit"
  did_holds,            # logical; explicit user declaration of D-ID
  did_condition,        # character scalar (NA_character_ if unconditional)
  stan_loglik_block,    # Stan snippet for the model block (per-observation
                        # target += ... ; references eta[i] and y_real[i] or
                        # y_int[i] per y_type)
  stan_log_lik_block,   # Stan snippet for generated quantities log_lik[i]
  stan_y_pred_block,    # Stan snippet for generated quantities y_pred[i]
  y_type,               # one of "real", "integer"
  did_reference         # citation supporting did_holds
)

Literal recipe for a custom log-Normal K = 1 family (a degenerate one-slot mirror of the lognormal_loc_scale pattern of §2.4):

my_family <- gdpar_family_custom(
  name               = "my_log_normal",
  link               = "log",
  did_holds          = TRUE,
  did_condition      = NA_character_,
  stan_loglik_block  =
    "target += normal_lpdf(log(y_real[i]) | eta[i], sigma_y[1]);",
  stan_log_lik_block =
    "log_lik[i] = normal_lpdf(log(y_real[i]) | eta[i], sigma_y[1]);",
  stan_y_pred_block  =
    "y_pred[i] = exp(normal_rng(eta[i], sigma_y[1]));",
  y_type             = "real",
  did_reference      = "User declaration"
)

The package emits an informational message restating the two user responsibilities (likelihood correctness and identifiability) every time a custom family is constructed. See ?gdpar_family_custom for the full Roxygen and Lemma 1B in Block 1 (§6.4) for the methodological backing of the D-ID declaration.

7. Known limitations and future work

K > 1 with grouping (J_groups > 1). Both coef.gdpar_fit and predict.gdpar_fit with newdata raise gdpar_unsupported_feature_error. In-sample prediction (newdata = NULL) is supported.
K > 1 with B-spline W on newdata. The in-sample path supports B-spline through apply_W_basis_diff() in Stan; the R-side reconstruction on new data presently mirrors only the polynomial branch.
Heterogeneous K = 3+. Queued for Session 8.4.
Hurdle continuous families (hurdle_gamma, hurdle_lognormal). Queued for sub-phase 8.3.7+ and Session 8.4.

References

Dunn, P. K., & Smyth, G. K. (1996). Randomized quantile residuals. Journal of Computational and Graphical Statistics, 5(3), 236–244.
Greene, W. H. (1994). Accounting for excess zeros and sample selection in Poisson and negative binomial regression models. NYU Stern Working Paper EC-94-10.
Hartig, F. (2024). DHARMa: Residual Diagnostics for Hierarchical (Multi-Level / Mixed) Regression Models. R package version 0.4.7.
Lambert, D. (1992). Zero-inflated Poisson regression, with an application to defects in manufacturing. Technometrics, 34(1), 1–14.
Mullahy, J. (1986). Specification and testing of some modified count data models. Journal of Econometrics, 33(3), 341–365.
Wood, S. N. (2017). Generalized Additive Models: An Introduction with R (2nd ed.). Chapman and Hall/CRC.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.

Distributional Regression K > 1 and Residual Diagnostics with DHARMa