vennLasso Intro
The vennLasso package is an implementation of the
methods proposed in Huling et al. (2018)
https://doi.org/10.1111/biom.12769. The underlying
methodology is motivated by the need to address population heterogeneity
in hospital system-wide risk modeling applications, however it can be
used in a wide variety of settings. It is useful for high-dimensional
modeling scenarios where heterogeneity is defined by several binary
factors which stratify the population into multiple subpopulations. For
example, vennLasso can be used in a hospital-wide risk
modeling application if covariate effects in risk models differ for
subpopulations of patients with different chronic conditions. Here the
chronic conditions are the binary stratifying factors. The
vennLasso provides computation for a variable selection
method which yields variable selection patterns which adhere to the
hierarchical nature of the relationships between the various
subpopulations.
If the chronic conditions congestive heart failure (CHF), chronic obstructive pulmonary disorder (COPD), and diabetes are used as the stratifying factors, the subpopulations may look like in Figure 1.
Figure 1: Sample sizes for each subpopulation in the motivating cohort.
Model Setup
We allow for covariate effects to vary based on a set of binary stratifying factors by positing separate (generalized) linear models for each subpopulation (defined by the presence of specific combinations of these binary factors). Denote \(Y_{ik}\) as the response for patient \(i\) of subpopulation \(k\), \(X_{ik}\) is the vector of length \(p_k\) of covariate values for patient \(i\) of subpopulation \(k\), and \(g(\cdot\) as a known link function. Continuing the example with models stratified based on CHF, COPD, and diabetes, the posited models are the following:
\[E[ Y_{ik}| \mathbf{X}_{ik}] = g^{-1}(\mathbf{X}_{ik}\boldsymbol{\beta}_{k,\bullet}), i = 1, \dots, n_k\],
where \(k \in \{ H, P, D, HP, HD, PD, HPD, none \}\),
\(H\) = Congestive Heart Failure
\(P\) = Chronic Obstructive Pulmonary Disease
\(D\) = Diabetes
\(HP\) = CHF + COPD
\(\dots\)
\(none\) = None of \(H\), \(P\), or \(D\),
\(\mathbf{X}_k\) is of dimension \(n_k\times p_k\), and \(\boldsymbol{\beta}_{k,\bullet}=(\beta_{k,1},\dots,\beta_{k,p})\). Note that different covariates are allowed for different subpopulations. This can be useful if there are variables specific to particular stratifying factors, e.g. the particular location of a heart failure is only relevant for patients with any heart failures.
The vennLasso package provides estimation and variable
selection for the parameters in these models. The variable selection is
performed in a scientifically-plausible manner that adheres to the
inherent relationships between the subpopulations. Furthermore, the
manner in which the variable selection is performed allows for the
borrowing of strength across subpopulations.
Borrowing Strength Across Subpopulations via Hierarchical Importance
Consider for a moment a simpler scenario where models are stratified
based on only CHF and diabetes. The variable selection performed by
vennLasso is based on an assumption of hierarchical
variable selection. This assumption has two components, outlined in
Figure 2 and for models with three stratifying factors in Figure 3. The
first component of the hierarchical assumption is that if a particular
variable is not important for a given subpopulation, it is not
important for all ‘descendent’ subpopulations, i.e. subpopulations that
only have any of the binary factors present in the given subpopulation.
For example, the \(P\) subpopulation is
a descendent of the \(HPD\)
subpopulation. The second component is that if a particular variable is
important for a given subpopulation, it should be important for all
‘parent’ subpopulations, i.e. any subpopulations that have at least all
of the stratifying factor present in the given subpopulation. For
example, the \(HPD\) subpopulation is a
parent of the \(HP\) subpopulation.
For the \(j^{th}\) variable
Figure 2: Hierarchical selection patterns for models with two stratifying factors.
Figure 3: The two highlighted groups represent hierarchical selection patterns.
Loss Function for Hierarchical Selection
The vennLasso package estimates coefficients with the
hierarchical variable selection patterns described above using the
penalized likelihood framework:
\[ f(\boldsymbol{\beta}) = \sum_{k=1}^K\ell_k(\boldsymbol{\beta}_{k,\bullet}) - \lambda P(\boldsymbol{\beta}) \qquad (1) \]
where \(\ell_k\) are log-likelihood functions (or negative loss), \(P\) is an overlapping group lasso penalty with special structure to induce hierarchical selection patterns, and \(\boldsymbol{\beta} = (\boldsymbol{\beta}_{H,\bullet},\boldsymbol{\beta}_{P,\bullet},\dots,\boldsymbol{\beta}_{HPD,\bullet}, \boldsymbol{\beta}_{none,\bullet})\) is the vector of all coefficients for all models. For simplicity here, we assume here that the number of variables is the same for each subpopulation, however the generalization to allow different variables for different subpopulations is straightforward and described in Section 2 of Huling et al. (2018).
The form of \(P\) is a group lasso penalty with overlapping groups:
\[ P(\boldsymbol{\beta}) = \sum_{j = 1}^p\sum_{G \in \color{red}{\mathcal{G}}}\lambda_{G,j}||\boldsymbol{\beta}_{G,j}||_2, \]
where \(\boldsymbol{\beta}_{G,j} \equiv \{\beta_{k,j}, k\in G \}\). The particular structure of the groups in \(\mathcal{G}\) determines patterns of selection. The group structure for models stratified on CHF, COPD, and diabetes is the following: \[\mathcal{G} = \{ \overline{{HPD}}, \overline{{HP}}, \overline{{HD}}, \overline{{PD}}, \overline{{H}}, \overline{{P}}, \overline{{D}}, none \}\]
- \(\overline{{HPD}} = \{ HPD, HP, HD, PD, H, P, D \}\)
- \(\overline{{HP}} = \{ HP, H, P \}\)
- \(\cdots\)
- \(\overline{{P}} = \{P \}\).
This group structure naturally generalizes to scenarios with an arbitrary number of stratifying factors. See Section 2 of Huling et al. (2018) for more details.
The vennLasso package minimizes (1) using an alternating
direction method of multipliers (ADMM) algorithm, described in the
Supplementary Material of Huling et al.
(2018); for the binomial family the ADMM iterations are nested
within Newton (iteratively reweighted least squares) steps.
