--- title: "Introduction to e2tree: Explainable Ensemble Trees" author: "Massimo Aria, Agostino Gnasso" date: "`r Sys.Date()`" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Introduction to e2tree: Explainable Ensemble Trees} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 5 ) ``` ## Overview The **e2tree** package implements the Explainable Ensemble Tree (E2Tree) methodology, which constructs a single interpretable decision tree that approximates the predictions of an ensemble model (such as a random forest). The key idea is to use the dissimilarity structure induced by the ensemble to guide the construction of the tree, thereby preserving the predictive information encoded in the ensemble while providing a transparent, human-readable model. The methodology is described in: - Aria, M., Gnasso, A., Iorio, C., & Pandolfo, G. (2024). Explainable ensemble trees. *Computational Statistics*, 39(1), 3--19. - Aria, M., Gnasso, A., Iorio, C., & Fokkema, M. (2026). Extending Explainable Ensemble Trees to Regression Contexts. *Applied Stochastic Models in Business and Industry*, 42(1), e70064. ## Installation ```{r eval = FALSE} # Install from CRAN (when available): install.packages("e2tree") # Or install the development version from GitHub: # devtools::install_github("massimoaria/e2tree") ``` ## Workflow A typical e2tree analysis follows these steps: 1. **Train an ensemble model** (random forest via `randomForest` or `ranger`) 2. **Compute the dissimilarity matrix** from the ensemble using `createDisMatrix()` 3. **Build the E2Tree** using `e2tree()` 4. **Inspect and visualize** the tree (print, summary, plot) 5. **Predict** on new data with `predict()` 6. **Validate** the E2Tree structure with `eValidation()` and `loi()` ## Classification Example We use the classic `iris` dataset to demonstrate a classification workflow. ### Step 1: Prepare data and train ensemble ```{r classification-setup} library(e2tree) data(iris) set.seed(42) smp_size <- floor(0.75 * nrow(iris)) train_ind <- sample(seq_len(nrow(iris)), size = smp_size) training <- iris[train_ind, ] validation <- iris[-train_ind, ] ``` ```{r classification-ensemble} if (!require("randomForest")) install.packages("randomForest", repos="https://cran.r-project.org") library(randomForest) ensemble <- randomForest(Species ~ ., data = training, importance = TRUE, proximity = TRUE) ensemble ``` ### Step 2: Compute dissimilarity matrix The ensemble induces a co-occurrence structure that captures how frequently pairs of observations fall into the same terminal nodes across the ensemble's trees. This structure is encoded in the co-occurrence matrix $\mathbf{O} = [o_{ij}]$, where each entry $o_{ij} \in [0, 1]$ reflects the weighted frequency with which observations $i$ and $j$ are grouped together. The corresponding dissimilarity matrix is $\mathbf{D} = \mathbf{1} - \mathbf{O}$. ```{r classification-dismatrix} D <- createDisMatrix(ensemble, data = training, label = "Species", parallel = list(active = FALSE, no_cores = 1)) dim(D) ``` ### Step 3: Build the E2Tree The `e2tree()` function constructs a single tree that best represents the ensemble's dissimilarity structure. The `setting` argument controls the stopping rules: - `impTotal`: minimum impurity to attempt a split - `maxDec`: minimum decrease in impurity required - `n`: minimum number of observations in a node - `level`: maximum tree depth ```{r classification-tree} setting <- list(impTotal = 0.1, maxDec = 0.01, n = 2, level = 5) tree <- e2tree(Species ~ ., training, D, ensemble, setting) ``` ### Step 4: Inspect the tree ```{r classification-print} print(tree) ``` ```{r classification-summary} summary(tree) ``` The `nodes()` accessor extracts the tree structure as a data frame: ```{r classification-nodes} # Terminal nodes only nodes(tree, terminal = TRUE) ``` ### Step 5: Visualization The `plot()` method renders the tree using `rpart.plot`: ```{r classification-plot, fig.width = 8, fig.height = 6} plot(tree, ensemble = ensemble, main = "E2Tree - Iris Classification") ``` You can also convert the tree to `rpart` or `partykit` formats for alternative visualizations: ```{r classification-rpart, fig.width = 8, fig.height = 6} # Convert to rpart rpart_obj <- as.rpart(tree, ensemble) # Convert to partykit (if installed) if (requireNamespace("partykit", quietly = TRUE)) { party_obj <- partykit::as.party(tree) plot(party_obj) } ``` ### Step 6: Prediction Use the standard `predict()` method on new data: ```{r classification-predict} pred <- predict(tree, newdata = validation) head(pred) ``` Fitted values for the training data: ```{r classification-fitted} head(fitted(tree)) ``` ### Step 7: Variable importance ```{r classification-vimp} vi <- vimp(tree, data = training) vi$vimp vi$g_imp ``` ## Regression Example The same workflow applies to regression problems. Here we use the `mtcars` dataset. ```{r regression-setup} data(mtcars) set.seed(123) smp_size <- floor(0.75 * nrow(mtcars)) train_ind <- sample(seq_len(nrow(mtcars)), size = smp_size) training_reg <- mtcars[train_ind, ] validation_reg <- mtcars[-train_ind, ] ``` ```{r regression-ensemble} ensemble_reg <- randomForest(mpg ~ ., data = training_reg, ntree = 500, importance = TRUE, proximity = TRUE) ``` ```{r regression-dismatrix} D_reg <- createDisMatrix(ensemble_reg, data = training_reg, label = "mpg", parallel = list(active = FALSE, no_cores = 1)) ``` ```{r regression-tree} setting_reg <- list(impTotal = 0.1, maxDec = 1e-6, n = 2, level = 5) tree_reg <- e2tree(mpg ~ ., training_reg, D_reg, ensemble_reg, setting_reg) print(tree_reg) ``` ```{r regression-plot, fig.width = 8, fig.height = 6} plot(tree_reg, ensemble = ensemble_reg, main = "E2Tree - MPG Regression") ``` For regression models, predictions include the standard deviation within each terminal node, providing a measure of prediction uncertainty: ```{r regression-predict} pred_reg <- predict(tree_reg, newdata = validation_reg) head(pred_reg) ``` Residuals are also available: ```{r regression-residuals} res <- residuals(tree_reg) summary(res) ``` ## Validation of the E2Tree Structure A fundamental question arises when using E2Tree: how well does the single interpretable tree capture the original ensemble's structure? The E2Tree reconstructs the ensemble's co-occurrence matrix $\mathbf{O}$ through its terminal node structure, producing an approximation $\hat{\mathbf{O}}_T$. Assessing the quality of this reconstruction is essential to ensure that the explanatory model faithfully represents the ensemble's decision logic. The `eValidation()` function provides two complementary approaches to validation, selectable through the `test` argument: - **`test = "mantel"`**: The Mantel test, a classical non-parametric test of *association* between similarity matrices. - **`test = "measures"`**: A family of divergence and similarity measures that assess *agreement* between the matrices, with permutation-based inference. - **`test = "both"`** (default): Performs both. ### Association vs. Agreement The distinction between these two approaches mirrors the classical distinction between *correlation* and *concordance* in method comparison studies (Bland & Altman, 1986; Lin, 1989). Two proximity matrices can be perfectly correlated ($r = 1$) and yet systematically disagree: if $\hat{\mathbf{O}}_T = c \cdot \mathbf{O}$ for any constant $c \neq 1$, the Mantel test would declare perfect association, yet the actual proximity values would not be faithfully reproduced. This distinction matters in the E2Tree context. The ensemble matrix $\mathbf{O}$ contains continuous values in $(0, 1)$, reflecting varying degrees of co-occurrence across the ensemble's trees. The E2Tree matrix $\hat{\mathbf{O}}_T$, by contrast, has a crisp block-diagonal structure: observations within the same terminal node share a common co-occurrence value, while observations in different nodes have zero co-occurrence. Even when the block structure perfectly matches the grouping pattern of $\mathbf{O}$, the element-wise values will differ because $\mathbf{O}$ is fuzzy and $\hat{\mathbf{O}}_T$ is crisp. A scale-sensitive divergence measure can quantify this discrepancy; a correlation-based measure cannot. ### The Mantel Test The Mantel test evaluates whether the E2Tree proximity matrix is *associated* with the ensemble proximity matrix. It computes a correlation-like statistic between the two matrices and assesses significance through permutation. ```{r val-mantel} val_mantel <- eValidation(training, tree, D, test = "mantel", graph = FALSE) print(val_mantel) ``` A significant Mantel test confirms that E2Tree preserves the overall *pattern* of the ensemble's proximity structure. However, it does not tell us how closely the actual proximity values are reproduced, nor does it indicate *where* the reconstruction fails. ### Divergence and Similarity Measures To assess *agreement* rather than mere association, we use a family of five measures, each capturing a distinct aspect of reconstruction quality: | Measure | Type | Range | What it captures | |---------|------|-------|------------------| | **nLoI** (normalized Loss of Interpretability) | divergence | [0, 1] | Scale-sensitive per-pair discrepancy with diagnostic decomposition | | **Hellinger** distance | divergence | [0, 1] | Variance-stabilized element-wise difference (a true metric) | | **wRMSE** (weighted Root Mean Squared Error) | divergence | [0, 1] | Weighted average discrepancy emphasizing high-proximity regions | | **RV** coefficient | similarity | [0, 1] | Global structural similarity (matrix-level correlation) | | **SSIM** (Structural Similarity Index) | similarity | [-1, 1] | Local spatial pattern similarity (luminance, contrast, structure) | For *divergence* measures, lower values indicate better reconstruction. For *similarity* measures, higher values indicate better reconstruction. Statistical significance is assessed through a unified permutation testing framework based on simultaneous row/column permutation of $\hat{\mathbf{O}}_T$. This preserves the block-diagonal structure of the E2Tree matrix while breaking the correspondence between observation labels and node assignments, providing the appropriate null hypothesis: under $H_0$, E2Tree groups observations randomly with respect to the ensemble structure. ```{r val-measures} val_meas <- eValidation(training, tree, D, test = "measures", graph = FALSE, n_perm = 499) print(val_meas) ``` All divergence measures are significantly lower than their null expectations (negative Z-statistics), while all similarity measures are significantly higher (positive Z-statistics), confirming that the E2Tree faithfully reconstructs the ensemble's proximity structure. ```{r val-measures-table} measures(val_meas) ``` ### Visualizing the Reconstruction The `plot()` method for `eValidation` objects provides a visual summary including heatmaps of both proximity matrices, the null distribution of nLoI (when permutation tests are performed), and the LoI decomposition: ```{r val-plot, fig.width = 8, fig.height = 7} val_full <- eValidation(training, tree, D, test = "both", graph = FALSE, n_perm = 499) plot(val_full) ``` ### The Normalized Loss of Interpretability (nLoI) Among the divergence measures, the nLoI deserves special attention because of its unique *decomposability* into within-node and between-node components. The nLoI is defined as: $$\text{nLoI}(\mathbf{O}, \hat{\mathbf{O}}_T) = \frac{1}{M} \sum_{i 0.3) indicates that E2Tree is separating pairs with substantial ensemble proximity. The practitioner should consider increasing the tree's complexity (more terminal nodes) or relaxing pruning constraints. - A **high `mean_in`** (e.g., > 0.1) indicates poor within-node calibration: the partition boundaries are well-placed, but the within-node proximity values deviate from the ensemble's. This typically reflects the inherent fuzzy-to-crisp structural transition. - **Low values of both** confirm that the E2Tree faithfully captures the ensemble structure --- the partition is well-placed and the calibration is accurate. This diagnostic capability is fundamentally impossible with the Mantel test or any other scalar association measure, which can only report that the overall reconstruction is "good" or "poor" without indicating the source of the discrepancy. ```{r loi-plot} plot(result) ``` #### Permutation Test for LoI The `loi_perm()` function provides a standalone permutation test for the nLoI, useful when a quick assessment of reconstruction significance is needed without computing all five measures: ```{r loi-perm} perm <- loi_perm(prox$ensemble, prox$e2tree, n_perm = 499, seed = 42) print(perm) plot(perm) ``` ### Validation of the Regression Example The same validation framework applies to regression E2Trees: ```{r val-regression} val_reg <- eValidation(training_reg, tree_reg, D_reg, test = "measures", graph = FALSE, n_perm = 499) print(val_reg) ``` Regression E2Trees typically show higher nLoI and Hellinger values than classification, consistent with the greater complexity of regression proximity structures and the typically smaller sample sizes. Nonetheless, all measures should achieve significant p-values, confirming that the reconstruction captures genuine structure. ## Comparison with Related Packages The **e2tree** package complements existing R infrastructure for tree-based models: - **partykit**: Provides generic classes for representing trees. E2Tree objects can be converted to `party` objects via `as.party()`, enabling the use of partykit's rich visualization and prediction infrastructure. - **stablelearner**: Explores the stability of ensemble trees via `stabletree()`. While stablelearner focuses on variable and cutpoint stability, e2tree constructs a single tree that captures the ensemble's dissimilarity-based structure --- a complementary perspective. - **rpart**: The standard recursive partitioning package. E2Tree objects can be converted to `rpart` objects via `as.rpart()` for visualization with `rpart.plot`. ## Session Info ```{r session-info} sessionInfo() ```