The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
p > 1
rigor = "full" vs
rigor = "fast"rigor = "full")The framework introduced in the overview vignette specifies that individual parameters \(\theta_i\) are decomposed as
\[\theta_i \;=\; \theta_{\text{ref}} + \Delta(x_i, \theta_{\text{ref}}).\]
Two foundational questions remain. First, what functional form does \(\Delta\) take? Second, when are the components \((\theta_{\text{ref}}, \Delta)\) recoverable from data?
This block addresses both questions with explicit separation of three logical layers, mixed in many treatments and easy to conflate:
The first layer is settled by Theorem 1A under a clearly stated linearity hypothesis on the function classes. The second layer is bridged by Lemma 1B, which makes explicit the requirement that the response distribution family be itself identifiable in its parameter. The third layer is captured by Proposition 1C, which connects identifiability in a chosen finite basis to non-singularity of an empirical Gram matrix. Approximation, the Bayesian extension, and the hypernetwork case are treated in further results, each with its own hypotheses and its own scope.
The treatment is for an applied statistician. Each formal statement is preceded by a non-technical paragraph that explains its content in plain language; each proof step is followed by a paragraph that interprets what the step achieves.
We work with \(n\) observational units indexed by \(i = 1, \ldots, n\). Each unit has:
The framework specifies \(\theta_i = \theta_{\text{ref}} + \Delta(x_i, \theta_{\text{ref}})\), where \(\theta_{\text{ref}} \in \Theta \subseteq \mathbb{R}^p\) is the population reference and \(\Delta : \mathcal{X} \times \Theta \to \mathbb{R}^p\) is the deviation function.
| Symbol | Domain | Meaning |
|---|---|---|
| \(x_i\) | \(\mathbb{R}^d\) | Individual covariates (centered; see (C1) below) |
| \(\theta_i\) | \(\mathbb{R}^p\) | Individual parameter vector |
| \(\theta_{\text{ref}}\) | \(\Theta \subseteq \mathbb{R}^p\) | Population reference parameter |
| \(\theta_0\) | \(\Theta\) | Anchor point of \(W\) (typically \(\theta_0 = \theta_{\text{ref}}\)) |
| \(\theta_*\) | \(\Theta\) | True value of \(\theta_{\text{ref}}\) in identifiability arguments |
| \(\Delta(x, \theta)\) | \(\mathbb{R}^p\) | Deviation function |
| \(\mathcal{D}(\theta)\) | — | Response distribution given \(\theta\) |
| \(\mu\) | — | Marginal distribution of \(X\) in the population |
| \(\pi_\Theta\) | — | Prior on \(\theta_{\text{ref}}\) in the Bayesian setting |
| \(\odot\) | — | Hadamard (elementwise) product |
| \(\otimes\) | — | Kronecker / tensor product |
The space of admissible deviation functions is infinite-dimensional. We stratify it into nested levels indexed by the joint polynomial order in \((x, \theta_{\text{ref}})\). Each level is a structurally distinct family of deviation functions, with its own expressive capacity and its own identifiability theorem.
The hierarchy is constructive: lower levels are subfamilies of higher ones. Whether the union of all polynomial-restricted levels is dense in the space of regular deviation functions is the subject of the approximation scheme of Section 3.8 —an approximation result, not a closed proposition, presented with explicit caveats.
\[\Delta(x, \theta_{\text{ref}}) \;=\; 0.\]
All individuals share \(\theta_i = \theta_{\text{ref}}\). No individuation; standard non-mixed regression. Joint order: \((0, 0)\).
\[\Delta(x, \theta_{\text{ref}}) \;=\; A x,\]
with \(A \in \mathbb{R}^{p \times d}\) fixed and not depending on \(\theta_{\text{ref}}\). This is the classical random-coefficient model and standard hierarchical model with covariates in random effects (Raudenbush and Bryk 2002). The framework’s distinguishing feature —reference-dependence of \(\Delta\)— is absent. Joint order: \((1, 0)\).
\[\boxed{\;\Delta(x, \theta_{\text{ref}}) \;=\; \underbrace{a(x)}_{\text{additive}} \;+\; \underbrace{b(x) \odot \theta_{\text{ref}}}_{\text{multiplicative (Hadamard)}} \;+\; \underbrace{W(\theta_{\text{ref}})\, x}_{\text{modulated}}.\;}\]
The components are:
Each component captures a distinct interaction between individual covariates and the population reference:
Level 2 is the smallest level extending Level 1 in which \(\Delta\) depends nontrivially on \(\theta_{\text{ref}}\).
The Hadamard product is replaced by a full matrix:
\[\Delta(x, \theta_{\text{ref}}) \;=\; a(x) \;+\; B(x)\, \theta_{\text{ref}} \;+\; W(\theta_{\text{ref}})\, x,\]
with \(B : \mathcal{X} \to \mathbb{R}^{p \times p}\). Level 2 is the special case \(B(x) = \operatorname{diag}(b(x))\).
Adds quadratic interactions:
\[\Delta(x, \theta_{\text{ref}}) \;=\; \Delta^{(2)}(x, \theta_{\text{ref}}) \;+\; \bigl[x^\top Q_k(\theta_{\text{ref}})\, x\bigr]_{k=1}^p \;+\; \bigl[\theta_{\text{ref}}^\top R_k(x)\, \theta_{\text{ref}}\bigr]_{k=1}^p \;+\; \theta_{\text{ref}} \odot W(\theta_{\text{ref}})\, x,\]
with \(\Delta^{(2)}\) the Level-2 part. The doubly-modulated term has joint order \((1, 2)\).
All polynomial-tensor terms up to total joint order \(K\) in \((x, \theta_{\text{ref}})\):
\[\Delta(x, \theta_{\text{ref}}) \;=\; \sum_{(j, k) : 1 \leq j + k \leq K,\, j \geq 1} \mathcal{T}_{j, k}\bigl(x^{\otimes j}, \theta_{\text{ref}}^{\otimes k}\bigr),\]
with \(\mathcal{T}_{j, k}\) multilinear of orders \((j, k)\). Terms with \(j = 0\) are excluded as non-individuating (they would collapse into a redefinition of \(\theta_{\text{ref}}\)).
\[\Delta(x, \theta_{\text{ref}}) \;=\; \Phi(x, \theta_{\text{ref}}),\]
with \(\Phi : \mathcal{X} \times \Theta \to \mathbb{R}^p\) an arbitrary measurable function subject to centering constraints. Realized in practice by neural networks of sufficient capacity (hypernetworks). Identifiability at this level is treated separately as Proposition 1F.
The hierarchy of Sections 3.1-3.6 stratifies admissible deviation forms by joint polynomial order in \((x, \theta_{\text{ref}})\), but does not require the components \(a, b, W\) themselves to be polynomial: Level 2 admits arbitrary measurable \(a, b\) and arbitrary continuous \(W\). The approximation result below applies to the polynomial restriction of the hierarchy, where \(a, b, W\) are themselves polynomials. We state it precisely and flag its limits.
Scheme 1D (Approximation by Polynomial AMM). Let \(\mathcal{X}, \Theta\) be compact subsets of \(\mathbb{R}^d, \mathbb{R}^p\) respectively, and let \(\Delta : \mathcal{X} \times \Theta \to \mathbb{R}^p\) be continuous. For every \(\varepsilon > 0\) there exists a finite \(K\) and a Level-\(K\) deviation function \(\Delta^{(K)}\) —with all sub-functions \(a, b, W\) themselves restricted to be polynomials of degree at most \(K\) in their arguments— such that \[\sup_{(x, \theta) \in \mathcal{X} \times \Theta} \bigl\|\Delta(x, \theta) - \Delta^{(K)}(x, \theta)\bigr\| < \varepsilon.\]
Argument. By the Stone-Weierstrass theorem, real polynomials are uniformly dense in \(C(\mathcal{X} \times \Theta, \mathbb{R})\). Approximating each coordinate of \(\Delta\) by a polynomial of total degree \(\leq K\) yields a Level-\(K\) AMM in which the components \(a, b, W\) are polynomial restrictions —a strictly smaller subfamily than the general Level-\(K\) that admits arbitrary measurable \(a, b\) and arbitrary continuous \(W\). The truncation error decreases monotonically as \(K\) grows. \(\square\)
Caveats explicitly flagged.
The scheme is an approximation result, not an identifiability theorem. Approximating \(\Delta\) by \(\Delta^{(K)}\) does not preserve identifiability: the components of \(\Delta^{(K)}\) may differ substantially from those of \(\Delta\) even when \(\sup |\Delta - \Delta^{(K)}|\) is small.
The scheme covers the polynomial restriction of Level-\(K\) AMM, not the full Level-\(K\) AMM with arbitrary measurable \(a, b\) and continuous \(W\). AMM with spline or kernel components, for instance, lies outside the polynomial restriction.
The scheme requires compactness of \(\mathcal{X}\) and \(\Theta\). Without compactness, Stone-Weierstrass density does not apply.
The error bound \(\varepsilon\) is uniform over the compact domain but says nothing about the rate at which \(K\) must grow. Practical truncation requires a separate analysis of the smoothness of \(\Delta\).
In summary: the hierarchy of polynomial-AMM levels is dense in the space of continuous deviation functions on compact domains, in the sense of uniform approximation. This is a useful fact for thinking about the framework’s expressive capacity, but it is not a closed identifiability statement. The identifiability content of the block lives in results 1A-1F below (Theorem 1A, Lemma 1B, Proposition 1C, Scheme 1D, Theorem 1E, Proposition 1F).
Level 0 and Level 1 do not realize the framework: in both, \(\Delta\) does not depend on \(\theta_{\text{ref}}\). Among the remaining levels, Level 2 has three properties that single it out as the canonical default:
(P1) Minimality of reference-dependence. Level 2 is the smallest extension of Level 1 in which \(\theta_{\text{ref}}\) enters \(\Delta\) nontrivially.
(P2) Separability of components. The three components \((a, b, W)\) correspond to three structurally distinct mechanisms with separate interpretations.
(P3) Computational tractability. Level 2 admits closed-form gradients, finite-dimensional sufficient statistics, and tractable Hamiltonian Monte Carlo sampling under standard regularity.
The identifiability theorem below is stated and proved for Level 2. Sections 7 and 8 sketch how it extends to higher levels and verifies that standard models in the literature fall within its scope.
We collect the assumptions used by the identifiability results. All six are interpretable and verifiable on data.
(C1) Centering of covariates. \(\mathbb{E}_\mu[X] = 0\).
(C2) Functional centering of \(a\). \(\mathbb{E}_\mu[a(X)] = 0\).
(C3) Functional centering of \(b\). \(\mathbb{E}_\mu[b(X)] = 0\).
(C4) Anchoring of the modulating matrix. There exists \(\theta_0 \in \Theta\) with \(W(\theta_0) = 0_{p \times d}\).
(C5) Support and integrability. \(\mathcal{X}\) is a Borel subset of \(\mathbb{R}^d\) with \(\mu(\mathcal{X}) = 1\); \(\mathbb{E}_\mu[\|X\|^2] < \infty\); \(\mathrm{Cov}_\mu(X)\) has full rank \(d\); \(a, b \in L^2(\mu, \mathbb{R}^p)\); \(W\) is continuous on \(\Theta\) with \(\sup_{\theta \in \Theta} \|W(\theta)\|_F < \infty\).
(C6) Non-degeneracy of the reference. The true population reference \(\theta_*\) has all coordinates non-zero, equivalently \(\theta_* \in \Theta^\circ := \{\theta \in \Theta : \theta_k \neq 0 \text{ for all } k = 1, \ldots, p\}\).
Joint consequence of (C1)-(C4) (used throughout): the population-averaged deviation vanishes for every \(\theta_{\text{ref}}\),
\[\mathbb{E}_\mu[\Delta(X, \theta_{\text{ref}})] = \mathbb{E}[a(X)] + \mathbb{E}[b(X)] \odot \theta_{\text{ref}} + W(\theta_{\text{ref}}) \mathbb{E}[X] = 0.\]
This is the centering of the framework.
(C6) is genuinely a non-trivial assumption. When \(\theta_*\) has a zero coordinate, the multiplicative component in that coordinate vanishes regardless of \(b\), and identifiability of \(b\) in that coordinate is lost. (C6) rules out this degeneracy. In practice, (C6) is satisfied for parameters that are bounded away from zero by their problem definition (rates, scales, precisions), and can be enforced by reparametrization (e.g., \(\log \theta_{\text{ref}}\) for positive parameters).
The identifiability question separates into three logically distinct levels, each requiring its own hypotheses.
(L1) Algebraic-functional identifiability. Given the latent function \(\theta_i(\cdot)\) on \(\mathrm{supp}(\mu)\), do its components \((\theta_{\text{ref}}, a, b, W)\) uniquely correspond to the function? This is a question about the algebraic structure of the AMM decomposition, independent of how data are generated.
(L2) Statistical identifiability. Given the observable joint distribution of \((X_i, Y_i)\) —not the latent \(\theta_i\)— do the components uniquely correspond to the data-generating process? This bridges (L1) to data via the response distribution \(\mathcal{D}(\theta)\).
(L3) Numerical verifiability. Given a fitted model in a finite parametric representation, can a computational diagnostic flag identifiability failure? This translates abstract conditions into runtime checks.
We treat each layer with its own theorem or proposition.
Two flavors of FIC are useful, distinguished by their scope.
Abstract FIC at \(\theta_*\). Let \(\mathcal{F}_a, \mathcal{F}_b \subseteq L^2_0(\mu, \mathbb{R}^p)\) and \(\mathcal{F}_W \subseteq C(\Theta, \mathbb{R}^{p \times d})\) be the function classes from which \(a, b, W\) are drawn. Define the subspaces
\[\mathcal{S}_a \;=\; \mathcal{F}_a, \qquad \mathcal{S}_b(\theta_*) \;=\; \bigl\{x \mapsto h(x) \odot \theta_* : h \in \mathcal{F}_b\bigr\}, \qquad \mathcal{S}_W \;=\; \bigl\{x \mapsto M\, x : M \in \mathbb{R}^{p \times d}\bigr\}.\]
Abstract FIC at \(\theta_*\). The three subspaces \(\mathcal{S}_a, \mathcal{S}_b(\theta_*), \mathcal{S}_W\) are linearly independent as subsets of \(L^2(\mu, \mathbb{R}^p)\): \[f_a + f_b + f_W = 0 \;\;\text{(in } L^2\text{)} \;\Longrightarrow\; f_a = 0,\; f_b = 0,\; f_W = 0,\] for \(f_a \in \mathcal{S}_a\), \(f_b \in \mathcal{S}_b(\theta_*)\), \(f_W \in \mathcal{S}_W\).
Basis-restricted FIC. When \(\mathcal{F}_a, \mathcal{F}_b\) are infinite-dimensional and only a finite truncation is parametrized in the implementation, FIC restricted to that truncation is a strictly weaker condition. Let \(B = (B_a, B_b)\) be a finite basis with \(B_a = \{\phi_j^{(a)}\}_{j=1}^{J_a}\) for \(\mathcal{F}_a\) and \(B_b = \{\phi_k^{(b)}\}_{k=1}^{J_b}\) for \(\mathcal{F}_b\).
Basis-restricted FIC (\(\text{FIC}_B\)) at \(\theta_*\). Abstract FIC restricted to functions expressible in the bases \(B_a, B_b\) for the additive and multiplicative components, with \(W(\theta_*)\) ranging over \(\mathbb{R}^{p \times d}\).
When \(\mathcal{F}_a, \mathcal{F}_b\) are themselves finite-dimensional and \(B\) exhausts them, \(\text{FIC}_B\) coincides with abstract FIC. When \(\mathcal{F}_a, \mathcal{F}_b\) are larger, \(\text{FIC}_B\) is strictly weaker than abstract FIC.
This is the algebraic core of the block.
Theorem 1A. Suppose:
- (LIN) \(\mathcal{F}_a, \mathcal{F}_b\) are finite-dimensional linear subspaces of \(L^2_0(\mu, \mathbb{R}^p)\), and \(\mathcal{F}_W\) is a finite-dimensional linear space of continuous matrix-valued functions on \(\Theta\), all closed under addition.
- The standing assumptions (C1)-(C5) hold for every admissible decomposition.
- (C6) holds at the true population reference \(\theta_*\).
- Abstract FIC at \(\theta_*\) holds.
Then the latent function \[\theta_i^*(\cdot) \;=\; \theta_* + a(\cdot) + b(\cdot) \odot \theta_* + W(\theta_*) \cdot,\] as an element of \(L^2(\mu, \mathbb{R}^p)\), uniquely determines \((\theta_*, a, b, W(\theta_*))\) within \(\Theta \times \mathcal{F}_a \times \mathcal{F}_b \times \{W(\theta_*) : W \in \mathcal{F}_W\}\).
Scope of identification regarding \(W\). Theorem 1A identifies the value \(W(\theta_*) \in \mathbb{R}^{p \times d}\) at the single point \(\theta_*\). It does not identify \(W\) as a function on \(\Theta\): at any \(\theta \neq \theta_*\), \(W(\theta)\) remains unconstrained by the latent function evaluated at the single reference \(\theta_*\). Functional identification of \(W\) on a Bayesian prior support is the subject of Theorem 1E and rests on additional hypotheses (continuity of \(W\), connectedness of the prior support).
Plain reading. Under the explicit hypothesis that the function classes are linear subspaces (so that differences of admissible components remain in the class), and under the explicit assumption that no coordinate of the true reference is zero, the latent function \(\theta_i^*(\cdot)\) contains all the information needed to recover the four components \((\theta_*, a, b, W(\theta_*))\) uniquely —with \(W\) pinned only at the single point \(\theta_*\). The proof proceeds in three steps with conceptual commentary at each.
Suppose two tuples \((\theta_*, a, b, W)\) and \((\theta_*', a', b', W')\), both admissible under (LIN) and the standing assumptions, produce the same latent function in \(L^2(\mu, \mathbb{R}^p)\):
\[\theta_* + a(x) + b(x) \odot \theta_* + W(\theta_*) x \;=\; \theta_*' + a'(x) + b'(x) \odot \theta_*' + W'(\theta_*') x \qquad (\mu\text{-a.s.}). \tag{$\ast$}\]
We must show \(\theta_* = \theta_*'\), \(a = a'\) in \(L^2(\mu)\), \(b = b'\) in \(L^2(\mu)\), and \(W(\theta_*) = W'(\theta_*')\).
Take expectation of both sides of \((\ast)\) with respect to \(X \sim \mu\). The two representations coincide \(\mu\)-a.s., so their expectations coincide. Using (C2), (C3), and (C1) —each providing a zero substitution—:
\[\mathbb{E}\bigl[\theta_* + a(X) + b(X) \odot \theta_* + W(\theta_*) X\bigr] \;=\; \theta_* + 0 + 0 + 0 \;=\; \theta_*.\]
By the same argument the right-hand-side expectation equals \(\theta_*'\). Hence \(\theta_* = \theta_*' =: \theta_*\).
Conceptual commentary on Step 1. This is the workhorse. Centering forces \(\theta_{\text{ref}}\) to coincide with the marginal mean of \(\theta_i(X)\). Two tuples producing the same latent function thus must share the same \(\theta_{\text{ref}}\). Without (C1)-(C4) this step fails and translation non-identifiability persists.
With \(\theta_* = \theta_*'\), equation \((\ast)\) simplifies. Subtracting the common \(\theta_*\) from both sides:
\[a(x) + b(x) \odot \theta_* + W(\theta_*) x \;=\; a'(x) + b'(x) \odot \theta_* + W'(\theta_*) x \qquad (\mu\text{-a.s.}).\]
Define
\[\delta_a(x) \;=\; a(x) - a'(x), \qquad \delta_b(x) \;=\; b(x) - b'(x), \qquad \delta_W \;=\; W(\theta_*) - W'(\theta_*).\]
By (LIN), the linear subspaces are closed under subtraction: \(\delta_a \in \mathcal{F}_a\), \(\delta_b \in \mathcal{F}_b\), and \(\delta_W \in \mathbb{R}^{p \times d}\). The constraint becomes
\[\delta_a(x) + \delta_b(x) \odot \theta_* + \delta_W \, x \;=\; 0 \qquad (\mu\text{-a.s.}). \tag{$\diamond$}\]
Conceptual commentary on Step 2. The differences \(\delta_a, \delta_b, \delta_W\) measure disagreement between the two competing decompositions. Equation \((\diamond)\) states that, when measured along the direction in which the latent function is observed, all three differences must sum to zero. The next step shows —under FIC— that each must be zero individually. The role of (LIN) here is essential: without linearity, \(\delta_a = a - a'\) might not lie in \(\mathcal{F}_a\), and the proof structure collapses.
By Step 2, \(\delta_a \in \mathcal{S}_a\), \(\delta_b \odot \theta_* \in \mathcal{S}_b(\theta_*)\), and \(\delta_W \cdot x \in \mathcal{S}_W\). Equation \((\diamond)\) says their sum is zero in \(L^2(\mu, \mathbb{R}^p)\).
Abstract FIC at \(\theta_*\) (Section 6.2) says these three subspaces are linearly independent in \(L^2(\mu, \mathbb{R}^p)\). The only way the sum can be zero is for each summand to be zero in \(L^2\):
\[\delta_a = 0,\qquad \delta_b \odot \theta_* = 0,\qquad \delta_W \, x = 0 \;\;\text{in } L^2(\mu, \mathbb{R}^p).\]
The first equation gives \(a = a'\) in \(L^2(\mu)\). The second equation, together with (C6) (no coordinate of \(\theta_*\) is zero), gives \(\delta_b = 0\) in \(L^2(\mu)\), hence \(b = b'\) in \(L^2(\mu)\). The third equation, together with (C5) (\(\mathrm{Cov}_\mu(X)\) full rank, so \(X\) spans \(\mathbb{R}^d\) in \(L^2\)), gives \(\delta_W = 0\), hence \(W(\theta_*) = W'(\theta_*)\). \(\square\)
Conceptual commentary on Step 3. FIC is the structural condition saying the three component spaces do not overlap. If they did, mass could be transferred between them while leaving the latent function unchanged —exactly the non-identifiability we rule out. (C6) is needed in the second equation: without it, a zero coordinate of \(\theta_*\) would make the multiplicative term vanish and \(b\) would be unidentified in that coordinate.
A constructive converse to Theorem 1A holds, but only when the function class \(\mathcal{F}_W\) is rich enough to realize arbitrary target values at the point \(\theta_*\). This richness is not implied by (LIN) alone, and the necessity proof relies on it as a separate, formal hypothesis in addition to (LIN) and (C1)-(C6).
(EVAL) Surjectivity of point evaluation at \(\theta_*\). The point-evaluation map \[E_{\theta_*} \;:\; \mathcal{F}_W \;\longrightarrow\; \mathbb{R}^{p \times d}, \qquad W \;\longmapsto\; W(\theta_*),\] is surjective onto \(\mathbb{R}^{p \times d}\).
(EVAL) is satisfied by the standard finite-dimensional choices of \(\mathcal{F}_W\) encountered in implementations: matrix-valued polynomial spaces of degree \(\geq 1\), matrix-valued spline spaces with sufficient knots, finite linear combinations of any basis spanning \(\mathbb{R}^{p \times d}\) at \(\theta_*\). It is not automatic from (LIN), which only requires \(\mathcal{F}_W\) to be a finite-dimensional linear space, without constraining its image at any single point. The library checks (EVAL) at the chosen anchor before fitting and aborts with a diagnostic message if it fails.
Proposition (Necessity of FIC). Assume:
- (LIN) and (C1)-(C6) hold.
- (EVAL) holds at \(\theta_*\) as defined above.
If Abstract FIC at \(\theta_*\) fails, then the parametrization \((\theta_*, a, b, W(\theta_*)) \in \Theta \times \mathcal{F}_a \times \mathcal{F}_b \times \mathbb{R}^{p \times d}\) is not identifiable: there exist two distinct admissible tuples producing the same latent function.
Proof. Failure of Abstract FIC means there exist \(h_a \in \mathcal{F}_a\), \(h_b \in \mathcal{F}_b\), and \(H_W \in \mathbb{R}^{p \times d}\), not all zero, satisfying
\[h_a(x) + h_b(x) \odot \theta_* + H_W \, x \;=\; 0 \;\;\text{in } L^2(\mu, \mathbb{R}^p).\]
By (EVAL), there exists \(\Psi \in \mathcal{F}_W\) with \(\Psi(\theta_*) = H_W\). Take any admissible \((\theta_*, a^*, b^*, W^*)\) and define \(a' = a^* + h_a\), \(b' = b^* + h_b\), \(W' = W^* + \Psi\). By (LIN), the perturbed components remain in \(\mathcal{F}_a, \mathcal{F}_b, \mathcal{F}_W\) respectively. By construction, \(W'(\theta_*) = W^*(\theta_*) + H_W\). Centering conditions (C2)-(C3) hold for \(a', b'\) because \(\mathcal{F}_a, \mathcal{F}_b \subseteq L^2_0\). The latent function is unchanged because the perturbation \(h_a(x) + h_b(x) \odot \theta_* + H_W \, x = 0\) cancels by the FIC failure equation. The two tuples differ at the level of \((a, b, W(\theta_*))\) but produce the same latent function. \(\square\)
Conceptual commentary on the necessity proof. Two hypotheses are doing distinct work in the construction. (LIN) lets us add perturbations to admissible components and remain inside the function classes (closure under addition). (EVAL) lets us pick a specific target shift \(H_W\) at the anchor point and realize it as the value of some \(W \in \mathcal{F}_W\) (surjectivity of point evaluation). Without (LIN), \(a' = a^* + h_a\) might fall outside \(\mathcal{F}_a\). Without (EVAL), there might be no \(W \in \mathcal{F}_W\) with the required value at \(\theta_*\). Both are formal conditions, both are checkable in the chosen finite parametric representation, and both are satisfied by the standard choices used in practice.
Theorem 1A identifies the latent function \(\theta_i(\cdot)\), which is not directly observed. The bridge to observable data requires a separate, explicit assumption on the response distribution.
Lemma 1B. Suppose:
- The hypotheses of Theorem 1A hold.
- (D-ID) The response distribution family \(\{\mathcal{D}(\theta) : \theta \in \mathbb{R}^p\}\) is identifiable in \(\theta\): \(\mathcal{D}(\theta) = \mathcal{D}(\theta')\) implies \(\theta = \theta'\).
Then the joint distribution of \((X_i, Y_i)\) uniquely determines \((\theta_*, a, b, W(\theta_*))\) within the parameter space of Theorem 1A.
Proof sketch. Under (D-ID), the conditional law \(\mathcal{L}(Y_i \mid X_i = x)\) uniquely determines \(\theta_i(x)\) for \(\mu\)-a.e. \(x\). Hence the latent function \(\theta_i(\cdot)\) is identifiable from the joint distribution of \((X_i, Y_i)\). By Theorem 1A, the components are then identified. \(\square\)
When (D-ID) holds.
(D-ID) is a hypothesis on the modelling setup, not a theorem of the framework. When (D-ID) is in doubt, the user must verify it for the specific response distribution before relying on Lemma 1B.
Corollary 1. Under (C1)-(C4), \(\mathbb{E}_\mu[\Delta(X, \theta_{\text{ref}})] = 0\) for every \(\theta_{\text{ref}} \in \Theta\).
Proof. Direct from the joint consequence of (C1)-(C4) noted in Section 5. \(\square\)
Plain reading. When an individual is “average” (covariates equal the population mean), the deviation is zero: the parameter coincides with the population reference. The Anchoring Property posited in the overview vignette is here a theorem.
This proposition translates \(\text{FIC}_B\) —basis-restricted FIC— into a runtime check. The translation is not equivalent to abstract FIC; it captures only what can be diagnosed within the chosen finite parametric representation.
Proposition 1C. Fix a finite basis \(B = (B_a, B_b)\) for the additive and multiplicative components, with \(B_a = \{\phi_j^{(a)}\}_{j=1}^{J_a}\) a basis of \(\mathcal{F}_a\) and \(B_b = \{\phi_k^{(b)}\}_{k=1}^{J_b}\) a basis of \(\mathcal{F}_b\). Define the extended design matrix at \(\theta_*\): \[\mathbf{Z}_n(\theta_*) \;=\; \bigl[\, \Phi_a(X_{1:n}), \;\; \Phi_b(X_{1:n}) \odot \theta_*, \;\; X_{1:n} \,\bigr],\] where \(\Phi_a(X_{1:n})\) stacks the basis evaluations of \(B_a\) over the \(n\) rows, similarly \(\Phi_b\), and \(X_{1:n}\) stacks the centered covariates. The matrix \(\mathbf{Z}_n(\theta_*)\) is a well-defined evaluation matrix in \(\mathbb{R}^{n p \times (J_a + J_b + d)}\) as soon as the bases are fixed and the covariates are observed; no further hypotheses are required for its construction.
Let \(\mathbf{G}_n(\theta_*) = n^{-1} \mathbf{Z}_n(\theta_*)^\top \mathbf{Z}_n(\theta_*)\) be the empirical Gram matrix. As \(n \to \infty\), \(\mathbf{G}_n(\theta_*) \to \mathbf{G}(\theta_*)\) in probability, where \(\mathbf{G}(\theta_*) = \mathbb{E}[\mathbf{Z}^\top \mathbf{Z}]\) is the population Gram matrix.
Then, for the chosen finite representation \(B\), \(\text{FIC}_B\) at \(\theta_*\) holds if and only if \(\mathbf{G}(\theta_*)\) is non-singular.
Proof. In the chosen finite representation \(B\), \(\text{FIC}_B\) at \(\theta_*\) is by definition the linear independence of the columns of the (population) extended design matrix as elements of \(L^2(\mu, \mathbb{R}^p)\). Linear independence of a finite collection in an inner-product space is equivalent to non-singularity of its Gram matrix. The equivalence is therefore an algebraic identity within the representation \(B\), not a statement about \(\mathcal{F}_a, \mathcal{F}_b, \mathcal{F}_W\) in their full abstract form. \(\square\)
Crucial caveats.
Proposition 1C diagnoses basis-restricted FIC, not abstract FIC. If \(\mathcal{F}_a, \mathcal{F}_b\) are infinite-dimensional and \(B\) is a finite truncation, abstract FIC may fail in directions outside \(B\)’s span and \(\mathbf{G}_n\) would not detect it.
The empirical Gram matrix \(\mathbf{G}_n\) approaches the population Gram matrix only as \(n \to \infty\). In finite samples, near-singularity (high condition number) is a warning, not a definitive failure of \(\text{FIC}_B\).
Proposition 1C implies non-identifiability within the basis \(B\). If \(\mathbf{G}_n\) is singular, the model parameters in basis \(B\) are not identifiable. The user can then either remove redundant basis functions or expand to a different basis.
Operational role. Before fitting, the implementation computes the smallest eigenvalue and condition number of \(\mathbf{G}_n(\theta_*)\) at the empirical mean of \(\theta\) (initialized via a preliminary fit) or at a user-specified anchor. If the smallest eigenvalue falls below a tolerance (default \(10^{-8}\)), fitting is aborted with a diagnostic output identifying which basis directions are linearly dependent. This is a necessary safeguard against basis-level non-identifiability, not a guarantee of abstract identifiability.
p > 1The Gram-matrix check of Proposition 1C is sufficient for scalar reference parameters \(\theta_{\text{ref}} \in \mathbb{R}\). When \(\theta_{\text{ref}} \in \mathbb{R}^p\) with \(p > 1\), the coord-wise factorization canonized in Phase F of Block 5.2 introduces a new failure mode that Proposition 1C cannot detect a priori: cross-coordinate aliasing between the per-coordinate additive component \(a_k\) and the shared modulating component \(W\). Condition C4-bis is the pre-fit structural guard against this failure mode, and the implementation backs it with a post-fit posterior-geometry diagnostic that the structural check cannot replace.
For \(p > 1\), the canonical AMM coordinate \(k\) contributes to the linear predictor as
\[\eta_{i,k} \;=\; \theta_{\text{ref}, k} \;+\; Z_{a,k}[i, \cdot] \, a_{\text{coef}, k} \;+\; \sum_{j=1}^{W_{\text{per\_k\_dim}}} \bigl(\theta_{\text{ref}, k}^{\,j} - \theta_{\text{anchor}, k}^{\,j}\bigr) \, W_{\text{raw}}[r_{k,j}, \cdot] \, X[i, \cdot]^\top + \cdots,\]
where \(Z_{a,k}\) is the
per-coordinate additive design matrix (with column names listed in
design$Z_a_names_list[[k]]) and \(X\) is the shared modulating-component
design matrix (with column names listed in design$X_names,
equal to the x_vars of the spec).
The cross-coordinate aliasing arises when \(Z_{a,k}\) and \(X\) share columns by name. Concretely:
suppose a coordinate \(k\) has
a_k = ~ x1 and the modulating component uses
x_vars = c("x1", "x2"). Then the column \(x_1\) appears both in \(Z_{a,k}\) (as a basis function of \(a_k\)) and in \(X\) (as a covariate of \(W \cdot x\)). A perturbation of the
form
\[a'_k(x) \;=\; a_k(x) + c \, x_1, \qquad W'_{k, x_1}(\theta) \;=\; W_{k, x_1}(\theta) - c / (\theta_k^1 - \theta_{\text{anchor}, k}^1)\]
leaves \(\eta_{i,k}\) invariant at the single value of \(\theta_{\text{ref}, k}\) used to evaluate the diagnostic, but the two components \((a_k, W)\) are different. Posterior geometry detects this aliasing as a divergent funnel between the two components; the pre-fit check should detect the structural overlap that makes the aliasing possible in the first place.
(C4-bis) Coord-wise structural disjointness. For every coordinate \(k = 1, \ldots, p\): \[\mathrm{names}\bigl(Z_{a,k}\bigr) \,\cap\, \mathrm{names}(X) \;=\; \emptyset.\]
Plain reading: at the column-name level, no covariate that enters the
additive component of coordinate \(k\)
may also appear in the modulating component’s x_vars. The
condition is conservative: structural overlap is
necessary for the cross-coordinate aliasing to occur,
but not sufficient (regularization or specific data configurations may
suppress the practical impact). The implementation defaults to flagging
overlap as a failure (rigor = "full") and offers a
permissive mode (rigor = "fast") that only emits a warning,
intended for users who have explicitly accepted the overlap with
informed regularization.
A natural-looking alternative would be to extend the Gram matrix of Proposition 1C with the \(W\)-block columns \((\theta_{\text{ref}, k}^{\,j} - \theta_{\text{anchor}, k}^{\,j}) \cdot X\) for each \(j = 1, \ldots, W_{\text{per\_k\_dim}}\) and check rank deficiency directly. This does not work as a pre-fit check, for two interlocking reasons:
The \(W\)-block columns are scalar multiples of \(X\) per coordinate \(k\) at a fixed \(\theta_{\text{ref}}\). Because the diagnostic evaluates the basis at the single point \(\theta_*\) (typically zero on the linear-predictor scale, possibly a small non-zero value if \(b\) is active), the \(W_{\text{per\_k\_dim}}\) columns per coordinate \(k\) collapse to scalar multiples of the same \(X\). The extended Gram matrix has rank \(\dim(Z_{a,k}) + d\) (where \(d = \dim(X)\)), not \(\dim(Z_{a,k}) + W_{\text{per\_k\_dim}} \cdot d\) as the algebraic check would naively expect. Rank deficit appears even when the components are perfectly identifiable, producing a false positive.
A Jacobian-based check at \((a = 0, W = 0)\) inherits the same defect. The information of \(\theta_{\text{ref}}\) in \(W \cdot x\) enters only through \(W \neq 0\); evaluating the Jacobian at \(W = 0\) erases the very signal the check is supposed to find. The defect is structural, not numerical.
The implementation therefore separates the two checks:
gdpar_check_identifiability() does internally for \(p > 1\) via the private helper
check_C4_bis_per_k().rigor = "full" vs
rigor = "fast"The check exposed by gdpar_check_identifiability() for
\(p > 1\) accepts an additional
argument rigor with two values:
gdpar_check_identifiability(amm = spec, data = df, rigor = "full") # default
gdpar_check_identifiability(amm = spec, data = df, rigor = "fast")The "full" mode is the default. Per coordinate \(k\):
tol (default \(10^{-8}\)). FAIL if rank-deficient.The aggregated report’s passed field is
TRUE only when every coordinate passes. The aggregation is
conservative by construction (any single per-coordinate failure flags
the overall report).
The "fast" mode runs the rank check but does
not fail on structural overlap; it emits a single consolidated
warning of class gdpar_c4bis_overlap_warning at the end of
the loop with the full list of overlapping coordinates. This mode is
intended for users who have explicitly accepted the overlap with
informed regularization (e.g., a tightly-informative prior on the \(W\) block of the overlapping covariate, or
domain knowledge that the multiplicative interaction is the relevant
signal).
The full per-coordinate breakdown is stored in
report$c4_bis$per_k and accessible via print()
or direct subsetting. Each per-\(k\)
entry records passed, lambda_min,
lambda_max, condition_number,
shared_cols, and collinear_directions.
rigor = "full")A scenario with deliberate column-name overlap between the additive
component of coordinate 1 and the modulating component’s
x_vars. The check correctly flags the structural overlap
and aborts the fit.
When called inside gdpar() with
skip_id_check = FALSE (the default), the failed check
aborts the fit with class gdpar_identifiability_error and
returns the report in the error data, so the user can inspect it
programmatically:
The remedy is straightforward: declare per-coordinate additive bases
that are disjoint from the modulating component’s
x_vars. The two coordinates can use different covariates in
\(a_k\); the modulating component
carries the cross-coordinate coupling without aliasing.
The two coordinates of \(a\) use
distinct sets of covariates (z1 for coord 1,
z2 for coord 2), and the modulating component uses
different covariates entirely (x1, x2). There
is no name-level overlap between \(Z_{a,k}\) and \(X\) for any coordinate, so C4-bis passes
structurally and the fit proceeds.
C4-bis is a pre-fit structural condition specific to \(p > 1\): column-level disjointness
between the per-coordinate additive design and the modulating-component
covariates. The check is necessary but not sufficient; the post-fit
posterior-geometry diagnostics (divergences, ESS, \(\widehat{R}\)) are the conclusive forensic
for \(\theta_{\text{ref}}\)-mediated
aliasing. The implementation defaults to the conservative
rigor = "full" mode that aborts on overlap, and offers
rigor = "fast" for the cases where overlap is genuinely
intended and regularized.
Block 6.5 promotes the scalar (or coord-wise) reference \(\theta_{\text{ref}}\) to a per-group anchor
\(\theta_{\text{ref}}[g]\), \(g = 1, \ldots, J_{\text{groups}}\), sampled
hierarchically from \(\mathrm{Normal}(\mu_{\theta_{\text{ref}}},
\sigma_{\theta_{\text{ref}}})\). The grouping is activated by the
user via the one-sided formula argument group of
gdpar() (for example group = ~ species).
Default group = NULL reduces bit-exactly to the Block 6
single-anchor regime.
The per-group anchor introduces a new structural failure mode that the Gram-matrix check of Section 6.6 (C1-C4) cannot detect, because that check operates on the centered design \((Z_a, Z_b \cdot \theta_{\text{ref}}, X)\) without conditioning on the group indicator. We isolate it as condition (C7) and the package enforces it pre-fit. The numbering follows the Block 6.5 sequence (C1-C6 are reserved for Block 1; see Appendix B); the symbol (C7) is distinct from the (C5) of Block 1 (Support and integrability) introduced in Section 5.
If \(Z_a\) (or \(Z_b\)) contains a column that is constant within every level of the grouping variable, that column is rank-deficient with the per-group dummy matrix \(G = \mathbf{1}_{\{g = j\}}\), \(j = 1, \ldots, J_{\text{groups}}\). The implied alias is:
\[ \theta_i \;=\; \theta_{\text{ref}}[g_i] + Z_a[i,\,]^{\top} a + \cdots \;=\; \theta_{\text{ref}}[g_i] + c \cdot \mathbf{1}_{g_i} + Z_a^{(-)}[i,\,]^{\top} a^{(-)} + \cdots, \]
where \(Z_a^{(-)}\) drops the aliased column and the constant \(c\) associated to that column is absorbed into \(\theta_{\text{ref}}[g_i]\). The model is point-non-identified along the ray \(\theta_{\text{ref}}[g] \mapsto \theta_{\text{ref}}[g] + c\), \(a \mapsto a - c \cdot e_{\text{column}}\), and HMC will exhibit divergences, low ESS, or chain-level drift.
The same failure mode arises for any indirect alias: \(\mathrm{rank}([G \mid Z_a]) < \mathrm{ncol}(G) + \mathrm{ncol}(Z_a)\), i.e. a non-trivial linear combination of \(Z_a\) columns reproduces a column of \(G\) even if no single column of \(Z_a\) is constant within each group. The Gram-matrix check of Section 6.6 does not see this because it does not include \(G\) in its block.
Condition (C7). When
use_groups = 1, neither \(Z_a\) nor \(Z_b\) may contain columns that are rank-deficient with the per-group indicator matrix \(G\). Equivalently, \(\mathrm{rank}([G \mid Z_a]) = \mathrm{ncol}(G) + \mathrm{ncol}(Z_a)\) and likewise for \(Z_b\).
(C7) extends Section 6.6 of Block 1 with one extra check that is
activated only when the user supplies group.
The pre-flight check .check_group_aliasing_c7() invoked
by gdpar() runs two layers:
tol.
This catches the direct alias factor(group) in
a or b, as well as any deterministic function
of group.qr(M, tol = tol).
This catches indirect aliases that the per-column check misses.A violation aborts with gdpar_input_error, naming the
offending columns. The user must either remove the aliased columns from
the AMM spec or fit without the group argument.
Defense in depth: the standard Block 1 identifiability check
gdpar_check_identifiability() runs before (C7) inside
gdpar(), and frequently catches the direct
factor(group) case (because \(Z_a\) centered is rank-deficient even
without conditioning on \(G\)). (C7)
covers the residual subtler aliases that pass the Block 1 check.
a fails (C7)(C7) is a single-layer extension of Block 1 specific to per-group
anchors. The package enforces it pre-fit via
.check_group_aliasing_c7(), complementing the Block 1
Gram-matrix check and the C4-bis cross-component check. Together they
form a three-tier pre-flight: (C1-C4) global rank, (C4-bis)
cross-component per coord, (C7) group-anchor anti-aliasing. The post-fit
forensic remains the posterior-geometry diagnostic (divergences, ESS,
\(\widehat{R}\)).
In the Bayesian setting, \(\theta_{\text{ref}}\) has a prior \(\pi_\Theta\) over \(\Theta\). Theorem 1A identifies \(W\) at the single point \(\theta_*\). Outside the support of the prior, no data information about \(W\) is accumulated; identifiability of \(W\) on \(\Theta\) as a whole is therefore not the right question. The right question is identifiability on \(\overline{\mathrm{supp}(\pi_\Theta)}\), the closure of the prior support. We state three tiers of identifiability, restricted to this set, with explicitly different hypotheses and explicitly different conclusions.
Theorem 1E. Suppose:
- The hypotheses of Theorem 1A hold at \(\pi_\Theta\)-almost every \(\theta_* \in \mathrm{supp}(\pi_\Theta)\) (in particular Abstract FIC holds \(\pi_\Theta\)-a.e.).
- The hypothesis (D-ID) of Lemma 1B holds.
Let \(W, W' \in \mathcal{F}_W\) be two candidates producing the same observable joint distribution of \((X, Y)\) for all admissible \(\theta_{\text{ref}}\) in the prior support. Then:
(a) Identification \(\pi_\Theta\)-almost everywhere. \(W(\theta) = W'(\theta)\) for \(\pi_\Theta\)-a.e. \(\theta \in \mathrm{supp}(\pi_\Theta)\).
(b) Identification in \(L^2(\pi_\Theta)\). \(W\) and \(W'\) are equal as elements of \(L^2(\pi_\Theta; \mathbb{R}^{p \times d})\).
(c) Identification in \(C(\overline{\mathrm{supp}(\pi_\Theta)}, \mathbb{R}^{p \times d})\). Suppose furthermore:
- (BAY-1) \(\mathrm{supp}(\pi_\Theta)\) is the closure of a connected open subset \(U \subseteq \Theta\) (so that \(\overline{U} = \mathrm{supp}(\pi_\Theta)\)).
- (BAY-2) \(W\) and \(W'\) are continuous on \(\overline{\mathrm{supp}(\pi_\Theta)}\) (this is part of (C5)).
- (BAY-3) \(\pi_\Theta\) assigns strictly positive measure to every non-empty open subset of \(U\), so that the \(\pi_\Theta\)-a.e. identification set is dense in \(U\). (Automatic when \(\pi_\Theta\) is absolutely continuous with positive density on \(U\).)
Then \(W\) and \(W'\) are equal as elements of \(C(\overline{\mathrm{supp}(\pi_\Theta)}, \mathbb{R}^{p \times d})\).
The conclusion does not extend to points outside \(\overline{\mathrm{supp}(\pi_\Theta)}\), where the data carry no information about \(W\).
Proof sketch.
For each \(\theta_*\) where Theorem 1A’s hypotheses hold (by assumption a \(\pi_\Theta\)-full-measure subset of \(\mathrm{supp}(\pi_\Theta)\)), Theorem 1A identifies \(W(\theta_*)\). Combined with (D-ID) via Lemma 1B, this gives \(W(\theta_*) = W'(\theta_*)\) at each such \(\theta_*\). Hence equality holds \(\pi_\Theta\)-a.e. on \(\mathrm{supp}(\pi_\Theta)\).
\(\pi_\Theta\)-a.e. equality of two elements of \(L^2(\pi_\Theta; \mathbb{R}^{p \times d})\) is equality in \(L^2(\pi_\Theta)\) by definition.
Two ingredients are needed to lift (a) to equality in the continuous-function space:
Combining (c1) and (c2) gives the conclusion in (c). \(\square\)
Plain reading. The most reliable conclusion of the theorem is tier (b): \(W\) is identified as an element of \(L^2(\pi_\Theta; \mathbb{R}^{p \times d})\) under just the basic hypotheses (Theorem 1A applied \(\pi_\Theta\)-a.e., plus (D-ID)). Tier (c) —identification as an element of \(C(\overline{\mathrm{supp}(\pi_\Theta)})\)— is a strictly stronger conclusion that requires three extra regularity assumptions: connected-open structure of the support ((BAY-1)), continuity of \(W\) ((BAY-2), already in (C5)), and full \(\pi_\Theta\)-loading of the interior ((BAY-3)) so that the \(\pi_\Theta\)-a.e. identification set is dense. The framework’s recommended default is to report identification at the \(L^2(\pi_\Theta)\) level, escalating to \(C(\overline{\mathrm{supp}(\pi_\Theta)})\) only when the three regularity conditions are explicitly verified for the chosen prior. For absolutely continuous priors with positive density on a connected open set, the three are automatic.
Caveats.
Tiers (a) and (b) are equivalence-class statements: \(W\) is determined except on a \(\pi_\Theta\)-null set. For prior support with isolated atoms or disconnected pieces, two functions agreeing as elements of \(L^2(\pi_\Theta)\) may differ pointwise on null subsets without violating the conclusion.
Tier (c) requires both a “fat” topological structure of the support ((BAY-1)) and density of the \(\pi_\Theta\)-identification set in the interior ((BAY-3)). (BAY-3) is the condition guaranteeing density; without it, one could have \(\pi_\Theta\)-a.e. equality with the identification set concentrated on a \(\pi_\Theta\)-charged subset that is not topologically dense, defeating the continuous-extension argument.
None of the three tiers identifies \(W\) outside \(\overline{\mathrm{supp}(\pi_\Theta)}\). At points \(\theta \notin \overline{\mathrm{supp}(\pi_\Theta)}\), the value \(W(\theta)\) is unconstrained by the data. Predictions at out-of-support reference values rely entirely on prior assumptions and should be reported as such.
The hypernetwork case requires a separate treatment because \(\mathcal{F}_a, \mathcal{F}_b, \mathcal{F}_W\) —the images of feedforward neural networks— are not linear subspaces of \(L^2(\mu, \mathbb{R}^p)\) but rather nonlinear manifolds. The hypothesis (LIN) of Theorem 1A does not apply, and the theorem’s conclusion does not transfer. We state what can and cannot be claimed.
Proposition 1F. Consider the hypernetwork model \[\theta_i \;=\; \theta_{\text{ref}} + a_\phi(x_i) + b_\phi(x_i) \odot \theta_{\text{ref}} + W_\phi(\theta_{\text{ref}}) x_i,\] where \(a_\phi, b_\phi, W_\phi\) are realized by feedforward neural networks with parameters \(\phi\), trained with a loss that explicitly enforces (C1)-(C4) via penalty terms (Section 10.3). Then:
(i) The realized function \(\Phi_\phi(x, \theta) = a_\phi(x) + b_\phi(x) \odot \theta + W_\phi(\theta) x\) is the object of inference, in the sense that all reported posteriors, predictions, and component decompositions are defined on \(\Phi_\phi\) as an element of the realized function class.
(ii) The network parameters \(\phi\) are not identifiable from \(\Phi_\phi\): standard symmetries of feedforward networks —permutation of hidden units, sign flips, weight rescaling under the corresponding nonlinearity— produce parameter tuples \(\phi \neq \phi'\) with \(\Phi_\phi = \Phi_{\phi'}\).
(iii) Under (D-ID) of Lemma 1B, \(\Phi_\phi\) is identifiable up to \(L^2(\mu \otimes \pi_\Theta)\)-equivalence: two parameter tuples \(\phi, \phi'\) producing the same observable joint distribution of \((X, Y)\) for \(\theta_{\text{ref}}\) ranging over \(\mathrm{supp}(\pi_\Theta)\) yield realized functions \(\Phi_\phi\) and \(\Phi_{\phi'}\) that coincide \(\mu \otimes \pi_\Theta\)-almost everywhere on \(\mathrm{supp}(\mu) \times \mathrm{supp}(\pi_\Theta)\), equivalently are equal as elements of \(L^2(\mu \otimes \pi_\Theta; \mathbb{R}^p)\).
Proof. Claim (ii) follows from the listed symmetries, which are well-documented invariances of feedforward networks. Claim (i) is the framework’s stipulated convention, made consistent with (ii). Claim (iii): under (D-ID), the conditional law of \(Y\) given \((X, \theta_{\text{ref}})\) uniquely determines \(\theta_i(X, \theta_{\text{ref}})\) at \(\mu \otimes \pi_\Theta\)-a.e. point of \(\mathrm{supp}(\mu) \times \mathrm{supp}(\pi_\Theta)\). Subtracting \(\theta_{\text{ref}}\) (a known input to \(\Phi_\phi\)) yields equality of \(\Phi_\phi\) and \(\Phi_{\phi'}\) at \(\mu \otimes \pi_\Theta\)-a.e. point of the product support, i.e., equality in \(L^2(\mu \otimes \pi_\Theta; \mathbb{R}^p)\). \(\square\)
Scope of (iii). This claim is a function-level identifiability statement about the data-generating regression function realized by the network, conditional on the response distribution being identifiable in \(\theta\). It is strictly weaker than identifiability of the AMM decomposition \(\Phi_\phi(x, \theta) = a_\phi(x) + b_\phi(x) \odot \theta + W_\phi(\theta) x\) into its three components: two distinct triples \((a_\phi, b_\phi, W_\phi)\) may yield the same \(\Phi_\phi\) in \(L^2(\mu \otimes \pi_\Theta)\) if the AMM decomposition fails an analogue of FIC at the function-class level. Decomposition identifiability for the hypernetwork is the open question discussed below.
Approximation context (not part of the proposition). By the universal-approximation results of Hornik (1991) and Pinkus (1999), as network width or depth grows, the realized function class is dense in \(C(\mathcal{K}, \mathbb{R}^p)\) on any compact \(\mathcal{K} \subseteq \mathcal{X} \times \Theta\). This is a statement about the architecture’s expressive capacity —specifically, its ability to approximate any target deviation function— and is logically distinct from identifiability. Density says that for any continuous \(\Phi\) there exists an approximating \(\phi\); it does not say that the data uniquely determine which \(\Phi\) the parameter \(\phi\) is approximating. The framework records this distinction explicitly to avoid the common conflation.
On function-level identifiability of \(\Phi_\phi\).
Identifiability of \(\Phi_\phi\) from the observable joint distribution of \((X_i, Y_i)\) is not stated as a theorem here. It requires hypotheses that go beyond the scope of this block:
A general sufficient-conditions theorem for function-level identifiability of \(\Phi_\phi\) in the universal AMM setting is an open question whose closure depends on ingredients from posterior-contraction theory of Bayesian neural networks that are not settled at the time of writing. Block 6 collects partial results known: consistency under the Neural Tangent Kernel regime (Jacot et al. 2018; Bach 2017), PAC-Bayes generalization bounds (Dziugaite and Roy 2017), and explicit recognition of the asymptotic theory of Bayesian neural networks as an active research area (Hron et al. 2020).
Operational consequence for the implementation. Path 3 reports posteriors over \(\Phi_\phi\) together with three baseline diagnostics: (1) the empirical norms \(\|a_\phi\|_{L^2(\mu)}\), \(\|b_\phi\|_{L^2(\mu)}\), \(\|W_\phi\|_{L^2(\pi_\Theta)}\) to detect dormant components; (2) cross-validated posterior predictive checks to detect undetected non-identifiability via predictive instability; (3) explicit comparison to Path 1 fits on the same data when Path 1 is feasible, since Path 1 admits the closed-form identifiability theory of results 1A-1E (Theorem 1A, Lemma 1B, Proposition 1C, Scheme 1D, Theorem 1E). When Path 3 diverges substantially from Path 1 on the same data, the discrimination question becomes operational: is the divergence richer structural heterogeneity that Path 1 cannot capture, or undetected non-identifiability in Path 3 that the regularization did not eliminate? The two possibilities have very different consequences for trust in the fit, so the framework provides an empirical protocol to discriminate between them.
When Path 3 diverges from Path 1 on the same data, the library runs a four-step protocol whose joint output discriminates the two scenarios. The protocol is empirical: it does not prove identifiability or non-identifiability —no finite-sample test can— but it accumulates evidence in either direction with explicit decision criteria.
Step 1: Stability across random seeds. Refit Path 3 with \(K \geq 5\) distinct random initializations of the network parameters \(\phi\), holding architecture, regularization, and data fixed. For each test point \(x\), compute the empirical standard deviation of the predicted \(\widehat{\theta}_i(x)\) across the \(K\) runs (across-run SD), and compare it with the within-run posterior standard deviation of \(\theta_i(x)\) averaged across runs (within-run SD).
Step 2: Out-of-sample comparison via stratified \(k\)-fold cross-validation. Compute \(\widehat{\text{elpd}}_\text{loo}^{(\text{cv})}\) for both Path 1 and Path 3 over a \(k\)-fold split (default \(k = 10\)), stratified by quantiles of \(x\). Compute the per-fold differences \(\Delta\widehat{\text{elpd}}_j = \widehat{\text{elpd}}_3^{(j)} - \widehat{\text{elpd}}_1^{(j)}\) and their mean and standard error across folds.
Step 3: Calibration of posterior predictive intervals. On the held-out folds of Step 2, compute the empirical coverage of nominal \((1 - \alpha)\) posterior predictive intervals for both paths, at \(\alpha \in \{0.05, 0.20, 0.50\}\).
Step 4: Component-wise decomposition of the divergence. Compute the per-test-point difference \(\Delta\theta(x) = \widehat{\theta}_3(x) - \widehat{\theta}_1(x)\) and decompose it via the AMM structure:
\[\Delta\theta(x) \;=\; \underbrace{\Delta a(x)}_{\text{additive piece}} + \underbrace{\Delta b(x) \odot \widehat{\theta}_{\text{ref}}}_{\text{multiplicative piece}} + \underbrace{\Delta W(\widehat{\theta}_{\text{ref}}) x}_{\text{modulated piece}}.\]
For each piece, compute its share of the total \(\|\Delta\theta\|_{L^2(\mu)}\) and compare with the empirical norm of the corresponding component in Path 3.
Combine the four diagnostics into a single recommendation:
| Step 1 | Step 2 | Step 3 | Step 4 | Verdict |
|---|---|---|---|---|
| Stable | Path 3 wins | Both calibrated | In active components | Richer structure: Path 3’s divergence is structural; trust Path 3 |
| Stable | Path 3 wins | Path 3 miscalibrated | Any | Mixed: Path 3 captures structure but uncertainty quantification untrustworthy; report point predictions only |
| Stable | Tied or Path 1 wins | Either | Any | No structural advantage: Path 1 is preferred for parsimony |
| Unstable | Any | Any | Any | Non-identifiability: Path 3’s \(\Phi_\phi\) is not a stable target; trust Path 1 |
| Any | Any | Both miscalibrated | Any | Model misspecified: revisit the response distribution \(\mathcal{D}\); neither path is trustworthy as-is |
The library reports the four diagnostics in a single comparison panel together with the row of the decision rule that the data triggers. The diagnostic output does not replace user judgment: an analyst with substantive prior knowledge may override the verdict (e.g., choosing Path 3 despite an “unstable” verdict if the application demands the expressive capacity and predictive stability is acceptable for the use case), but the verdict makes the trade-off explicit rather than silent.
Why this protocol works. The four diagnostics target the four ways Path 3 can diverge from Path 1: (1) parameter-level convergence to inconsistent functional realizations (Step 1), (2) overfitting in-sample noise that does not generalize (Step 2), (3) producing point predictions that are correct on average but with wrong uncertainty (Step 3), (4) hiding the divergence in a component that the model has effectively zeroed out (Step 4). At least one of these will register when the divergence is non-identifiability; none will register when the divergence is genuine structural heterogeneity. The diagnostic is a necessary set of empirical filters, not a closed mathematical equivalence, but in practice it identifies the source of divergence reliably.
What it cannot do. No empirical protocol can prove identifiability with finite data. The protocol provides evidence weighted toward one or the other interpretation; it does not provide a formal certificate. In particular, a Path 3 fit can pass all four diagnostics and still be non-identifiable in directions that the held-out data do not probe —a fundamental limit of any data-based discrimination, not a defect of the protocol.
Results 1A-1F (Theorem 1A, Lemma 1B, Proposition 1C, Scheme 1D, Theorem 1E, Proposition 1F) are stated for the Level-2 canonical AMM. Higher levels admit analogous results under minimal modifications.
Level 2.5 (full-matrix multiplicative). Replace \(b(x) \odot \theta_*\) by \(B(x) \theta_*\), with \(B \in \mathcal{F}_B \subseteq L^2_0(\mu, \mathbb{R}^{p \times p})\) a finite-dimensional linear subspace. The proof of Theorem 1A goes through with the obvious modification of \(\mathcal{S}_b(\theta_*) = \{B \theta_* : B \in \mathcal{F}_B\}\). The Gram matrix gains \(J_B \cdot p\) columns instead of \(J_b \cdot p\).
Level 3 (quadratic extensions). Each new term contributes its own component subspace to the FIC. The condition becomes the joint linear independence of the (now five or more) subspaces. The Gram matrix has additional blocks; the eigenvalue diagnostic extends naturally. Theorem 1A’s proof structure is identical with extra terms in equations \((\ast)\) and \((\diamond)\).
Level \(K\) (polynomial closure). The FIC condition becomes joint linear independence of the \(O(K^2)\) component subspaces. The Gram matrix has \(O(K^4)\) entries; for \(K \leq 4\), common in applications, the diagnostic remains computationally tractable.
Level \(\infty\) (universal AMM, hypernetwork). Treated by Proposition 1F: parameter non-identifiability, density (not identifiability) from universal approximation, and explicit acknowledgement that function-level identifiability of \(\Phi_\phi\) requires further problem-specific hypotheses beyond this block.
We verify that each standard model in the literature falls within the scope of Theorem 1A by checking that its function classes satisfy (LIN).
Restriction: \(\mathcal{F}_a = \{0\}\), \(\mathcal{F}_b = \{0\}\), \(\mathcal{F}_W = \{0\}\). AMM Level: 0.
(LIN) trivially satisfied. Identifiability holds trivially.
Restriction: \(\mathcal{F}_a = \{x \mapsto Ax : A \in \mathbb{R}^{p \times d}\}\), \(\mathcal{F}_b = \{0\}\), \(\mathcal{F}_W = \{0\}\). AMM Level: 1.
(LIN) holds: \(\mathcal{F}_a\) is a linear subspace of dimension \(pd\). Identifiability under Theorem 1A reduces to non-singularity of \(\mathrm{Cov}_\mu(X)\), which is (C5).
Restriction: \(\mathcal{F}_a\) = spline space of finite dimension with no-constant basis; \(\mathcal{F}_b = \{0\}\); \(\mathcal{F}_W\) = matrix-valued spline space anchored at \(\theta_0\). AMM Level: 2 with \(b \equiv 0\).
(LIN) holds when the spline spaces have explicit bases. Identifiability under Theorem 1A reduces to FIC restricted to the chosen splines, diagnosed by Proposition 1C.
Restriction: \(\mathcal{F}_a\) = parametric finite-dim space (e.g., polynomials), \(\mathcal{F}_b\) = parametric finite-dim space, \(\mathcal{F}_W = \{0\}\). AMM Level: 2 with \(W \equiv 0\).
(LIN) holds. Identifiability under Theorem 1A reduces to FIC between \(\mathcal{F}_a\) and \(\mathcal{F}_b \odot \theta_*\). (C6) is critical: a zero coordinate of \(\theta_*\) destroys identifiability of \(b\) in that coordinate.
All three components active, finite-dim linear subspaces. AMM Level: 2 (full).
(LIN) holds. Identifiability requires the full FIC plus (C6).
Function classes are images of neural networks. (LIN) does not hold: these are nonlinear manifolds.
Theorem 1A does not apply. The hypernetwork case is treated by Proposition 1F: parameter non-identifiability, density (not identifiability) from universal approximation, and explicit acknowledgement that function-level identifiability of \(\Phi_\phi\) requires additional problem-specific hypotheses not stated as a closed theorem at this level of generality.
Theorem 1A identifies the components of a Level-2 AMM under FIC. The complementary practical question is which components are active: in a given dataset, \(a, b\), or \(W\) may be effectively zero.
For each restriction \(M_S\) with \(S \subseteq \{a, b, W\}\), fit and compute the leave-one-out expected log predictive density (Vehtari, Gelman, and Gabry 2017):
\[\widehat{\mathrm{elpd}}_{M_S} \;=\; \sum_{i=1}^n \log \widehat{p}_{-i}(y_i \mid x_i, M_S).\]
Compare candidates by \(\Delta\widehat{\mathrm{elpd}}\) relative to the full Level-2 model. Differences exceeding two standard errors are taken as substantive.
A novel diagnostic is stratified PSIS-LOO: \(\widehat{\mathrm{elpd}}\) is computed within subgroups defined by quantiles or categorical levels of \(x\). Components active only in subgroups indicate localizable structural heterogeneity. The library reports stratified PSIS-LOO automatically.
In the varying-coefficient implementation, generalized likelihood ratio tests adapted for spline models (Wood 2017) compare nested AMM restrictions, with \(\chi^2\) reference distribution on the difference in effective degrees of freedom.
In the hypernetwork implementation, post-training norms
\[\bigl\|a_\phi\bigr\|_{L^2(\mu)}, \quad \bigl\|b_\phi\bigr\|_{L^2(\mu)}, \quad \bigl\|W_\phi\bigr\|_{L^2(\pi_\Theta)}\]
are compared against the regularization scale. A norm close to zero indicates the corresponding component does not contribute and can be pruned.
The package produces, for any fitted model, a comparison table over the eight AMM restrictions \(M_\emptyset, M_a, M_b, M_W, M_{ab}, M_{aW}, M_{bW}, M_{abW}\), with \(\Delta\widehat{\mathrm{elpd}}\) and standard error per restriction.
Three principles follow from the theorems:
Stan declares \(\theta_{\text{ref}}\), the basis coefficients of \(a\) and \(b\), and the matrix coefficients of \(W\). (C2) and (C3) are enforced empirically by column-wise centering of the design matrices \(\mathbf{Z}_a\) and \(\mathbf{Z}_b\) inside the R-side AMM design constructor: each column of \(\mathbf{Z}_a\) (resp. \(\mathbf{Z}_b\)) has its sample mean subtracted, so that \(\mathrm{colMeans}(\mathbf{Z}_a) = \mathbf{0}\) (resp. \(\mathbf{Z}_b\)) exactly. Because \(\mathbb{E}_\mu[a(X)] = \mathrm{colMeans}(\mathbf{Z}_a) \cdot \mathbf{a}_{\mathrm{coef}}\) under the empirical \(\mu\), the centering of \(\mathbf{Z}_a\) alone satisfies (C2) for any choice of \(\mathbf{a}_{\mathrm{coef}} \in \mathbb{R}^{J_a}\); analogously for (C3). No further restriction on the basis coefficients is imposed by the Stan model, since imposing \(\mathbf{1}^\top \mathbf{a}_{\mathrm{coef}} = 0\) on top of column-centered \(\mathbf{Z}_a\) would arbitrarily restrict \(\mathbf{a}_{\mathrm{coef}}\) to a \((J_a - 1)\)-dimensional subspace and exclude effects with non-zero coefficient mean (e.g. \(\mathbf{a}_{\mathrm{coef}} = (1.0, -0.8)\) for two continuous covariates), with no support in (C2). (C4) is enforced by the parametrization \(W(\theta) = W_0(\theta) - W_0(\theta_0)\). The Gram-matrix check from Proposition 1C runs on the design matrix before sampling.
The varying-coefficient component is fit via penalized splines using
mgcv::gam. The AMM structure is encoded as
s(x, by = ...) interactions with a custom design matrix
satisfying the centering constraints. The smoothness penalty regularizes
the implicit basis coefficients consistent with Fan-Zhang
asymptotics.
The hypernetwork has three parallel sub-networks. (C1)-(C4) are enforced via penalty terms in the loss:
\[\mathcal{L}(\phi) \;=\; -\sum_{i=1}^n \log p(y_i \mid \theta_i) + \lambda_a \bigl\|\bar{a}_\phi\bigr\|^2 + \lambda_b \bigl\|\bar{b}_\phi\bigr\|^2 + \lambda_W \bigl\|W_\phi(\theta_0)\bigr\|_F^2,\]
with \(\bar{a}_\phi = n^{-1} \sum_i a_\phi(x_i)\). As \(\lambda_a, \lambda_b, \lambda_W \to \infty\), the centering and anchoring conditions are imposed exactly; in practice \(\lambda \in [10, 100]\) suffices. Identifiability holds at the function level (Proposition 1F).
This block has established, with explicit separation of three layers of identifiability:
The AMM hierarchy (Section 3) of structurally distinct deviation forms, with Scheme 1D giving an approximation result for the polynomial restriction and explicit caveats on its scope.
The canonical Level-2 AMM form (Section 3.3) and three reasons for its default status (Section 4).
The standing assumptions (C1)-(C6) (Section 5), all interpretable and verifiable, with (C6) elevated from a remark to an explicit hypothesis.
Theorem 1A (Section 6.3): algebraic-functional identifiability of the canonical AMM under the linearity of the function classes (LIN), the centering and anchoring conditions, the non-degeneracy of the reference (C6), and Abstract FIC at the true reference.
Lemma 1B (Section 6.4): the statistical bridge from the latent function to observable data, requiring the explicit hypothesis (D-ID) that the response distribution family is identifiable in its parameter.
Corollary 1 (Section 6.5): the centering of the framework as a theorem.
Proposition 1C (Section 6.6): the Gram-matrix diagnostic for basis-restricted FIC, with explicit caveats distinguishing it from abstract FIC.
Theorem 1E (Section 6.7): three-tier identifiability of \(W\) on the closure of the prior support —\(\pi_\Theta\)-a.e., \(L^2(\pi_\Theta)\), and pointwise on \(\overline{\mathrm{supp}(\pi_\Theta)}\) under additional regularity (BAY-1), (BAY-2), and (BAY-3)— with explicit notion of identifiability at each tier and explicit acknowledgement that the conclusion does not extend outside the prior support.
Proposition 1F (Section 6.8): the hypernetwork case, stating that \(\Phi_\phi\) is the object of inference, that the parameters \(\phi\) are not identifiable due to network symmetries, and that under (D-ID) the realized function \(\Phi_\phi\) is identifiable up to \(L^2(\mu \otimes \pi_\Theta)\)-equivalence —a function-level claim strictly weaker than identifiability of the AMM decomposition, which remains an open question at this level of generality.
Empirical discrimination protocol (Section 6.8.1) of four steps —stability across random seeds, out-of-sample comparison, calibration, component-wise divergence decomposition— with an explicit decision rule (Section 6.8.2) that weighs Path 3’s divergence from Path 1 toward “richer structural heterogeneity” or “undetected non-identifiability”. The protocol is acknowledged as evidence-weighted, not as a formal identifiability certificate.
Subsumption (Section 8) of standard models as special cases of Theorem 1A, with verification that (LIN) holds in each. The hypernetwork case is verified to fall outside Theorem 1A’s scope and into Proposition 1F’s scope.
A selection protocol for AMM components (Section 9) and concrete implementation implications for each of the three estimation paths (Section 10).
The three layers are separated throughout: algebraic-functional identifiability (Theorem 1A) is independent of the response distribution and is recovered by Lemma 1B at the statistical level; basis-restricted identifiability (Proposition 1C) is the runtime diagnostic, distinct from the abstract condition; the hypernetwork case is treated by its own proposition, not as a corollary of the linear-class result, and is kept honest about the gap between density and identifiability.
| Symbol | Meaning |
|---|---|
| \(\Delta(x, \theta_{\text{ref}})\) | Deviation function |
| \(a(x)\) | Additive component (vector \(p\)) |
| \(b(x)\) | Multiplicative component (vector \(p\), Hadamard with \(\theta_{\text{ref}}\)) |
| \(B(x)\) | Matrix multiplicative component (Level 2.5) |
| \(W(\theta)\) | Modulating component (matrix \(p \times d\)) |
| \(\theta_0\) | Anchor point in \(\Theta\) where \(W(\theta_0) = 0\) |
| \(\theta_*\) | True value of the population reference |
| \(\Theta^\circ\) | \(\{\theta \in \Theta : \theta_k \neq 0 \;\forall k\}\) |
| \(\mathcal{F}_a, \mathcal{F}_b, \mathcal{F}_W\) | Function classes for the components |
| \(\mathcal{S}_a, \mathcal{S}_b(\theta_*), \mathcal{S}_W\) | Component subspaces in \(L^2(\mu, \mathbb{R}^p)\) |
| \(B\) | Finite basis in a parametric representation |
| \(\mathbf{Z}_n(\theta_*)\) | Extended design matrix (Proposition 1C) |
| \(\mathbf{G}_n(\theta_*)\) | Empirical Gram matrix |
| \(\odot\) | Hadamard (elementwise) product |
| \(\otimes\) | Kronecker / tensor product |
| Hypothesis | Content |
|---|---|
| (C1) | \(\mathbb{E}_\mu[X] = 0\) |
| (C2) | \(\mathbb{E}_\mu[a(X)] = 0\) |
| (C3) | \(\mathbb{E}_\mu[b(X)] = 0\) |
| (C4) | \(W(\theta_0) = 0\) for some \(\theta_0 \in \Theta\) |
| (C5) | \(\mathrm{Cov}_\mu(X)\) full rank; \(a, b \in L^2\); \(W \in C(\Theta)\) bounded |
| (C6) | \(\theta_* \in \Theta^\circ\) (no zero coordinates) |
| (C7) | Per-group anchor anti-aliasing (Block 6.5, Section
6.6.2): when use_groups = 1, \(\mathrm{rank}([G \mid Z_a]) = \mathrm{ncol}(G) +
\mathrm{ncol}(Z_a)\) and likewise for \(Z_b\) |
| (LIN) | \(\mathcal{F}_a, \mathcal{F}_b, \mathcal{F}_W\) are finite-dim linear subspaces |
| (EVAL) | Point evaluation \(E_{\theta_*}: \mathcal{F}_W \to \mathbb{R}^{p \times d}\) is surjective (used in Necessity Proposition only) |
| (D-ID) | \(\{\mathcal{D}(\theta) : \theta \in \mathbb{R}^p\}\) identifiable in \(\theta\) |
| (BAY-1) | \(\mathrm{supp}(\pi_\Theta)\) is the closure of a connected open subset of \(\Theta\) |
| (BAY-2) | \(W\) continuous on \(\overline{\mathrm{supp}(\pi_\Theta)}\) (subsumed by (C5)) |
| (BAY-3) | \(\pi_\Theta\) assigns positive measure to every non-empty open subset of its interior |
Bach, F. (2017). Breaking the curse of dimensionality with convex neural networks. Journal of Machine Learning Research, 18, 1–53.
Dziugaite, G. K., and Roy, D. M. (2017). Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. Proceedings of UAI.
Fan, J., and Zhang, W. (2008). Statistical methods with varying coefficient models. Statistics and Its Interface, 1(1), 179–195.
Greene, W. H. (1994). Accounting for excess zeros and sample selection in Poisson and negative binomial regression models. NYU Working Paper EC-94-10.
Hastie, T., and Tibshirani, R. (1993). Varying-coefficient models. Journal of the Royal Statistical Society, Series B, 55(4), 757–796.
Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural Networks, 4(2), 251–257.
Lambert, D. (1992). Zero-inflated Poisson regression with an application to defects in manufacturing. Technometrics, 34(1), 1–14.
Patton, A. J. (2006). Modelling asymmetric exchange rate dependence. International Economic Review, 47(2), 527–556.
Pinkus, A. (1999). Approximation theory of the MLP model in neural networks. Acta Numerica, 8, 143–195.
Raudenbush, S. W., and Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods. Sage.
Teicher, H. (1963). Identifiability of finite mixtures. Annals of Mathematical Statistics, 34, 1265–1269.
Vehtari, A., Gelman, A., and Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27, 1413–1432.
Wood, S. N. (2017). Generalized Additive Models: An Introduction with R, 2nd ed. Chapman and Hall/CRC.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.