1 Introduction

Welocme at Champion Challenger analysys report. In table below we present detected models and labels.

label class type
LM WrappedModel Champion
RF WrappedModel Challenger
GBM WrappedModel Challenger

2 Funnel plot

Plot shown below helps us inspect our model performance across differenct categories. OY axis is row partitioned by column names of parition_data specified when creating object. Then whole test data is splited by quantilles of every column, one by one. Thanks to that we have subsets for which measure of Champion and every Challenger is being caluclated. Dots seen on the plot show difference in those measures, colours let us to distinguish variables. Thanks to measure property, that lower score value indicates better performance, dots spoted on the right side of vertical line means that Champion predcits following subset better than Challenger by value of measure shown on the OX axis. Accordingly dot on the left side of that line, means that Challanger was better by absolute value shown on the OX.

## $challanger_RF

## 
## $challanger_GBM

3 Overall comparison

In this section we present comparison of our models accross whole dataset.

3.1 Radar plot

Each score value has the property that lower score value indicates better model. Therfore some of those are inversed. Scores are scaled by dividing them by the max score, so they fit radar aesthetics.

label name value
LM mae 2.63e+02
GBM mae 1.37e+02
RF mae 2.05e+02
LM mse 8.01e+04
GBM mse 2.93e+04
RF mse 7.28e+04
LM rec 2.63e+02
GBM rec 1.37e+02
RF rec 2.04e+02
LM rroc 3.24e+12
GBM rroc 1.18e+12
RF rroc 2.95e+12

3.2 Accordance plot

Plot shows relation between Champion response, which is located on te OX axis and Challenger response placed at OY axis. Colour let us determinate exact Challenger label. The \(y = x\) is an accordance line and points located on it indicate on obseravtions where Champion and Challenger fully agree.

4 iBreakDown LM

5 iBreakDown RF

6 iBreakDown GBM

7 Trainig-test comparison

Following plot can help us determinate if our model encounter overfitting problem. OX axis stands for measure acquired on the train set while OY axis for measure acquired on test set. Therefore plot shows realation between measure on test and measure on trainig data. \(y = x\) line stands for a standard. Depends on type of measure, dot deeply below the line my indicate that model overfits, if measure has property that highest value indicates better performance. Overfit models can be also seen above the line, if measure has decreasing property.

8 Session Info

## R version 3.6.1 (2019-07-05)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=Polish_Poland.1250  LC_CTYPE=Polish_Poland.1250   
## [3] LC_MONETARY=Polish_Poland.1250 LC_NUMERIC=C                  
## [5] LC_TIME=Polish_Poland.1250    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] ggplot2_3.2.1     iBreakDown_0.9.9  DALEXtra_0.2.0    DALEX_0.4.9      
## [5] mlr_2.14.0        ParamHelpers_1.12
## 
## loaded via a namespace (and not attached):
##  [1] reticulate_1.13     shape_1.4.4         gbm_2.1.5          
##  [4] xfun_0.9            tidyselect_0.2.5    purrr_0.3.3        
##  [7] splines_3.6.1       lattice_0.20-38     colorspace_1.4-1   
## [10] htmltools_0.3.6     yaml_2.2.0          survival_2.44-1.1  
## [13] XML_3.98-1.20       rlang_0.4.1         pillar_1.4.2       
## [16] withr_2.1.2         glue_1.3.1          foreach_1.4.7      
## [19] stringr_1.4.0       munsell_0.5.0       gtable_0.3.0       
## [22] evaluate_0.14       codetools_0.2-16    labeling_0.3       
## [25] knitr_1.24          parallelMap_1.4     parallel_3.6.1     
## [28] auditor_1.0.0       highr_0.8           Rcpp_1.0.3         
## [31] scales_1.0.0        backports_1.1.5     checkmate_1.9.4    
## [34] jsonlite_1.6        gridExtra_2.3       fastmatch_1.1-0    
## [37] digest_0.6.22       packrat_0.5.0       stringi_1.4.3      
## [40] BBmisc_1.11         dplyr_0.8.3         ggrepel_0.8.1      
## [43] grid_3.6.1          tools_3.6.1         magrittr_1.5       
## [46] lazyeval_0.2.2      glmnet_3.0          tibble_2.1.3       
## [49] randomForest_4.6-14 ggdendro_0.1-20     crayon_1.3.4       
## [52] pkgconfig_2.0.3     MASS_7.3-51.4       Matrix_1.2-17      
## [55] data.table_1.12.2   hnp_1.2-6           assertthat_0.2.1   
## [58] rmarkdown_1.15      rstudioapi_0.10     iterators_1.0.12   
## [61] R6_2.4.0            compiler_3.6.1