The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

A-quick-tour-of-StMoE

Introduction

StMoE (Skew-t Mixture-of-Experts) provides a flexible and robust modelling framework for heterogenous data with possibly skewed, heavy-tailed distributions and corrupted by atypical observations. StMoE consists of a mixture of K skew-t expert regressors network (of degree p) gated by a softmax gating network (of degree q) and is represented by:

Model estimation/learning is performed by a dedicated expectation conditional maximization (ECM) algorithm by maximizing the observed data log-likelihood. We provide simulated examples to illustrate the use of the model in model-based clustering of heterogeneous regression data and in fitting non-linear regression functions.

It was written in R Markdown, using the knitr package for production.

See help(package="meteorits") for further details and references provided by citation("meteorits").

Application to a simulated dataset

Generate sample

n <- 500 # Size of the sample
alphak <- matrix(c(0, 8), ncol = 1) # Parameters of the gating network
betak <- matrix(c(0, -2.5, 0, 2.5), ncol = 2) # Regression coefficients of the experts
sigmak <- c(0.5, 0.5) # Standard deviations of the experts
lambdak <- c(3, 5) # Skewness parameters of the experts
nuk <- c(5, 7) # Degrees of freedom of the experts network t densities
x <- seq.int(from = -1, to = 1, length.out = n) # Inputs (predictors)

# Generate sample of size n
sample <- sampleUnivStMoE(alphak = alphak, betak = betak, sigmak = sigmak, 
                          lambdak = lambdak, nuk = nuk, x = x)
y <- sample$y

Set up StMoE model parameters

K <- 2 # Number of regressors/experts
p <- 1 # Order of the polynomial regression (regressors/experts)
q <- 1 # Order of the logistic regression (gating network)

Set up EM parameters

n_tries <- 1
max_iter <- 1500
threshold <- 1e-5
verbose <- TRUE
verbose_IRLS <- FALSE

Estimation

stmoe <- emStMoE(X = x, Y = y, K, p, q, n_tries, max_iter, 
                 threshold, verbose, verbose_IRLS)
## EM - StMoE: Iteration: 1 | log-likelihood: -358.341886247016
## EM - StMoE: Iteration: 2 | log-likelihood: -322.659951557355
## EM - StMoE: Iteration: 3 | log-likelihood: -320.693470431902
## EM - StMoE: Iteration: 4 | log-likelihood: -318.61107077321
## EM - StMoE: Iteration: 5 | log-likelihood: -316.212273528493
## EM - StMoE: Iteration: 6 | log-likelihood: -313.383764444746
## EM - StMoE: Iteration: 7 | log-likelihood: -310.065053962983
## EM - StMoE: Iteration: 8 | log-likelihood: -306.224283800106
## EM - StMoE: Iteration: 9 | log-likelihood: -301.880368911868
## EM - StMoE: Iteration: 10 | log-likelihood: -297.136415870862
## EM - StMoE: Iteration: 11 | log-likelihood: -292.198038917438
## EM - StMoE: Iteration: 12 | log-likelihood: -287.324695300884
## EM - StMoE: Iteration: 13 | log-likelihood: -282.746661707539
## EM - StMoE: Iteration: 14 | log-likelihood: -278.614147365268
## EM - StMoE: Iteration: 15 | log-likelihood: -274.996730286112
## EM - StMoE: Iteration: 16 | log-likelihood: -271.895041279085
## EM - StMoE: Iteration: 17 | log-likelihood: -269.280635577131
## EM - StMoE: Iteration: 18 | log-likelihood: -267.09149388565
## EM - StMoE: Iteration: 19 | log-likelihood: -265.271026320903
## EM - StMoE: Iteration: 20 | log-likelihood: -263.759092494974
## EM - StMoE: Iteration: 21 | log-likelihood: -262.502946815773
## EM - StMoE: Iteration: 22 | log-likelihood: -261.454038406018
## EM - StMoE: Iteration: 23 | log-likelihood: -260.574638252184
## EM - StMoE: Iteration: 24 | log-likelihood: -259.832146800618
## EM - StMoE: Iteration: 25 | log-likelihood: -259.201264232419
## EM - StMoE: Iteration: 26 | log-likelihood: -258.66209266237
## EM - StMoE: Iteration: 27 | log-likelihood: -258.200304380158
## EM - StMoE: Iteration: 28 | log-likelihood: -257.803020700055
## EM - StMoE: Iteration: 29 | log-likelihood: -257.459647457927
## EM - StMoE: Iteration: 30 | log-likelihood: -257.161109980419
## EM - StMoE: Iteration: 31 | log-likelihood: -256.902374475734
## EM - StMoE: Iteration: 32 | log-likelihood: -256.677764013955
## EM - StMoE: Iteration: 33 | log-likelihood: -256.481647583747
## EM - StMoE: Iteration: 34 | log-likelihood: -256.309532304488
## EM - StMoE: Iteration: 35 | log-likelihood: -256.157703550254
## EM - StMoE: Iteration: 36 | log-likelihood: -256.023316001022
## EM - StMoE: Iteration: 37 | log-likelihood: -255.903871921378
## EM - StMoE: Iteration: 38 | log-likelihood: -255.797646853338
## EM - StMoE: Iteration: 39 | log-likelihood: -255.702858065201
## EM - StMoE: Iteration: 40 | log-likelihood: -255.618465633147
## EM - StMoE: Iteration: 41 | log-likelihood: -255.543420022665
## EM - StMoE: Iteration: 42 | log-likelihood: -255.476395670569
## EM - StMoE: Iteration: 43 | log-likelihood: -255.416291553121
## EM - StMoE: Iteration: 44 | log-likelihood: -255.362320737453
## EM - StMoE: Iteration: 45 | log-likelihood: -255.313803352974
## EM - StMoE: Iteration: 46 | log-likelihood: -255.270074418416
## EM - StMoE: Iteration: 47 | log-likelihood: -255.230463560675
## EM - StMoE: Iteration: 48 | log-likelihood: -255.194449430696
## EM - StMoE: Iteration: 49 | log-likelihood: -255.161613595902
## EM - StMoE: Iteration: 50 | log-likelihood: -255.131581608645
## EM - StMoE: Iteration: 51 | log-likelihood: -255.104031659776
## EM - StMoE: Iteration: 52 | log-likelihood: -255.078686030772
## EM - StMoE: Iteration: 53 | log-likelihood: -255.055302250778
## EM - StMoE: Iteration: 54 | log-likelihood: -255.033928970051
## EM - StMoE: Iteration: 55 | log-likelihood: -255.01434724664
## EM - StMoE: Iteration: 56 | log-likelihood: -254.996347083498
## EM - StMoE: Iteration: 57 | log-likelihood: -254.979838144307
## EM - StMoE: Iteration: 58 | log-likelihood: -254.96469757084
## EM - StMoE: Iteration: 59 | log-likelihood: -254.950764911511
## EM - StMoE: Iteration: 60 | log-likelihood: -254.937902022156
## EM - StMoE: Iteration: 61 | log-likelihood: -254.925987986507
## EM - StMoE: Iteration: 62 | log-likelihood: -254.914882798003
## EM - StMoE: Iteration: 63 | log-likelihood: -254.904535254373
## EM - StMoE: Iteration: 64 | log-likelihood: -254.894864826624
## EM - StMoE: Iteration: 65 | log-likelihood: -254.885800788666
## EM - StMoE: Iteration: 66 | log-likelihood: -254.877280881681
## EM - StMoE: Iteration: 67 | log-likelihood: -254.869245760273
## EM - StMoE: Iteration: 68 | log-likelihood: -254.861646904019
## EM - StMoE: Iteration: 69 | log-likelihood: -254.854441631407
## EM - StMoE: Iteration: 70 | log-likelihood: -254.847592025442
## EM - StMoE: Iteration: 71 | log-likelihood: -254.841064308203
## EM - StMoE: Iteration: 72 | log-likelihood: -254.834828294085
## EM - StMoE: Iteration: 73 | log-likelihood: -254.828856906542
## EM - StMoE: Iteration: 74 | log-likelihood: -254.823125745702
## EM - StMoE: Iteration: 75 | log-likelihood: -254.81761269621
## EM - StMoE: Iteration: 76 | log-likelihood: -254.812297566296
## EM - StMoE: Iteration: 77 | log-likelihood: -254.807161750446
## EM - StMoE: Iteration: 78 | log-likelihood: -254.80218785685
## EM - StMoE: Iteration: 79 | log-likelihood: -254.797403497463
## EM - StMoE: Iteration: 80 | log-likelihood: -254.792855394281
## EM - StMoE: Iteration: 81 | log-likelihood: -254.788511319515
## EM - StMoE: Iteration: 82 | log-likelihood: -254.784341459075
## EM - StMoE: Iteration: 83 | log-likelihood: -254.780317790363
## EM - StMoE: Iteration: 84 | log-likelihood: -254.77641350682
## EM - StMoE: Iteration: 85 | log-likelihood: -254.772602954156
## EM - StMoE: Iteration: 86 | log-likelihood: -254.768863405824
## EM - StMoE: Iteration: 87 | log-likelihood: -254.76517125295
## EM - StMoE: Iteration: 88 | log-likelihood: -254.761506012367
## EM - StMoE: Iteration: 89 | log-likelihood: -254.757837544911
## EM - StMoE: Iteration: 90 | log-likelihood: -254.754161748511
## EM - StMoE: Iteration: 91 | log-likelihood: -254.750435218567
## EM - StMoE: Iteration: 92 | log-likelihood: -254.746629465947
## EM - StMoE: Iteration: 93 | log-likelihood: -254.742714669617
## EM - StMoE: Iteration: 94 | log-likelihood: -254.738659578476
## EM - StMoE: Iteration: 95 | log-likelihood: -254.734490817481
## EM - StMoE: Iteration: 96 | log-likelihood: -254.730192157155
## EM - StMoE: Iteration: 97 | log-likelihood: -254.725719853109
## EM - StMoE: Iteration: 98 | log-likelihood: -254.721031436972
## EM - StMoE: Iteration: 99 | log-likelihood: -254.716086365243
## EM - StMoE: Iteration: 100 | log-likelihood: -254.710846824594
## EM - StMoE: Iteration: 101 | log-likelihood: -254.705278656842
## EM - StMoE: Iteration: 102 | log-likelihood: -254.699352342126
## EM - StMoE: Iteration: 103 | log-likelihood: -254.693043955812
## EM - StMoE: Iteration: 104 | log-likelihood: -254.686335997371
## EM - StMoE: Iteration: 105 | log-likelihood: -254.679217983225
## EM - StMoE: Iteration: 106 | log-likelihood: -254.671686704349
## EM - StMoE: Iteration: 107 | log-likelihood: -254.663746075242
## EM - StMoE: Iteration: 108 | log-likelihood: -254.655406541352
## EM - StMoE: Iteration: 109 | log-likelihood: -254.646684061509
## EM - StMoE: Iteration: 110 | log-likelihood: -254.637598732378
## EM - StMoE: Iteration: 111 | log-likelihood: -254.628173166331
## EM - StMoE: Iteration: 112 | log-likelihood: -254.618430731211
## EM - StMoE: Iteration: 113 | log-likelihood: -254.608428384836
## EM - StMoE: Iteration: 114 | log-likelihood: -254.598186505404
## EM - StMoE: Iteration: 115 | log-likelihood: -254.587718089114
## EM - StMoE: Iteration: 116 | log-likelihood: -254.577032799573
## EM - StMoE: Iteration: 117 | log-likelihood: -254.566137142869
## EM - StMoE: Iteration: 118 | log-likelihood: -254.555035337377
## EM - StMoE: Iteration: 119 | log-likelihood: -254.543730791611
## EM - StMoE: Iteration: 120 | log-likelihood: -254.532228075951
## EM - StMoE: Iteration: 121 | log-likelihood: -254.520535176582
## EM - StMoE: Iteration: 122 | log-likelihood: -254.508665705371
## EM - StMoE: Iteration: 123 | log-likelihood: -254.496640769654
## EM - StMoE: Iteration: 124 | log-likelihood: -254.484490215664
## EM - StMoE: Iteration: 125 | log-likelihood: -254.472253026858
## EM - StMoE: Iteration: 126 | log-likelihood: -254.45997675758
## EM - StMoE: Iteration: 127 | log-likelihood: -254.447715961625
## EM - StMoE: Iteration: 128 | log-likelihood: -254.435565924281
## EM - StMoE: Iteration: 129 | log-likelihood: -254.423607788127
## EM - StMoE: Iteration: 130 | log-likelihood: -254.411895190203
## EM - StMoE: Iteration: 131 | log-likelihood: -254.400476823631
## EM - StMoE: Iteration: 132 | log-likelihood: -254.389393202635
## EM - StMoE: Iteration: 133 | log-likelihood: -254.37867404923
## EM - StMoE: Iteration: 134 | log-likelihood: -254.368336569733
## EM - StMoE: Iteration: 135 | log-likelihood: -254.358385555638
## EM - StMoE: Iteration: 136 | log-likelihood: -254.348812910957
## EM - StMoE: Iteration: 137 | log-likelihood: -254.339597972166
## EM - StMoE: Iteration: 138 | log-likelihood: -254.330710646008
## EM - StMoE: Iteration: 139 | log-likelihood: -254.322113858071
## EM - StMoE: Iteration: 140 | log-likelihood: -254.313766153391
## EM - StMoE: Iteration: 141 | log-likelihood: -254.305624190977
## EM - StMoE: Iteration: 142 | log-likelihood: -254.297627316357
## EM - StMoE: Iteration: 143 | log-likelihood: -254.289754304995
## EM - StMoE: Iteration: 144 | log-likelihood: -254.281967130667
## EM - StMoE: Iteration: 145 | log-likelihood: -254.274231823909
## EM - StMoE: Iteration: 146 | log-likelihood: -254.266518680729
## EM - StMoE: Iteration: 147 | log-likelihood: -254.258802087261
## EM - StMoE: Iteration: 148 | log-likelihood: -254.251060056927
## EM - StMoE: Iteration: 149 | log-likelihood: -254.243273588205
## EM - StMoE: Iteration: 150 | log-likelihood: -254.235425944923
## EM - StMoE: Iteration: 151 | log-likelihood: -254.227501935458
## EM - StMoE: Iteration: 152 | log-likelihood: -254.219487223213
## EM - StMoE: Iteration: 153 | log-likelihood: -254.211367644405
## EM - StMoE: Iteration: 154 | log-likelihood: -254.203128451916
## EM - StMoE: Iteration: 155 | log-likelihood: -254.1947533597
## EM - StMoE: Iteration: 156 | log-likelihood: -254.18622324363
## EM - StMoE: Iteration: 157 | log-likelihood: -254.177514369866
## EM - StMoE: Iteration: 158 | log-likelihood: -254.168596074928
## EM - StMoE: Iteration: 159 | log-likelihood: -254.159427917496
## EM - StMoE: Iteration: 160 | log-likelihood: -254.149956481982
## EM - StMoE: Iteration: 161 | log-likelihood: -254.140112305156
## EM - StMoE: Iteration: 162 | log-likelihood: -254.129807983281
## EM - StMoE: Iteration: 163 | log-likelihood: -254.118939720844
## EM - StMoE: Iteration: 164 | log-likelihood: -254.107396862317
## EM - StMoE: Iteration: 165 | log-likelihood: -254.095087362962
## EM - StMoE: Iteration: 166 | log-likelihood: -254.08198912314
## EM - StMoE: Iteration: 167 | log-likelihood: -254.068227669384
## EM - StMoE: Iteration: 168 | log-likelihood: -254.054144889453
## EM - StMoE: Iteration: 169 | log-likelihood: -254.040276614933
## EM - StMoE: Iteration: 170 | log-likelihood: -254.027150867197
## EM - StMoE: Iteration: 171 | log-likelihood: -254.015516270311
## EM - StMoE: Iteration: 172 | log-likelihood: -254.005428715919
## EM - StMoE: Iteration: 173 | log-likelihood: -253.996597298353
## EM - StMoE: Iteration: 174 | log-likelihood: -253.988680128159
## EM - StMoE: Iteration: 175 | log-likelihood: -253.982000739415
## EM - StMoE: Iteration: 176 | log-likelihood: -253.976458951876
## EM - StMoE: Iteration: 177 | log-likelihood: -253.971941901404
## EM - StMoE: Iteration: 178 | log-likelihood: -253.968330643806
## EM - StMoE: Iteration: 179 | log-likelihood: -253.965506908767
## EM - StMoE: Iteration: 180 | log-likelihood: -253.963358402817

Summary

stmoe$summary()
## ------------------------------------------
## Fitted Skew t Mixture-of-Experts model
## ------------------------------------------
## 
## StMoE model with K = 2 experts:
## 
##  log-likelihood df       AIC      BIC      ICL
##       -253.9634 12 -265.9634 -291.251 -291.283
## 
## Clustering table (Number of observations in each expert):
## 
##   1   2 
## 254 246 
## 
## Regression coefficients:
## 
##     Beta(k = 1) Beta(k = 2)
## 1   -0.00186094 -0.08398201
## X^1  2.57239459 -2.55715595
## 
## Variances:
## 
##  Sigma2(k = 1) Sigma2(k = 2)
##      0.4328539     0.6125303

Plots

Mean curve

stmoe$plot(what = "meancurve")

Confidence regions

stmoe$plot(what = "confregions")

Clusters

stmoe$plot(what = "clusters")

Log-likelihood

stmoe$plot(what = "loglikelihood")

Application to real dataset

Load data

library(MASS)
data("mcycle")
x <- mcycle$times
y <- mcycle$accel

Set up StMoE model parameters

K <- 4 # Number of regressors/experts
p <- 2 # Order of the polynomial regression (regressors/experts)
q <- 1 # Order of the logistic regression (gating network)

Set up EM parameters

n_tries <- 1
max_iter <- 1500
threshold <- 1e-5
verbose <- TRUE
verbose_IRLS <- FALSE

Estimation

stmoe <- emStMoE(X = x, Y = y, K, p, q, n_tries, max_iter, 
                 threshold, verbose, verbose_IRLS)
## EM - StMoE: Iteration: 1 | log-likelihood: -598.565702033975
## EM - StMoE: Iteration: 2 | log-likelihood: -586.594322769658
## EM - StMoE: Iteration: 3 | log-likelihood: -585.173591847768
## EM - StMoE: Iteration: 4 | log-likelihood: -582.369299935532
## EM - StMoE: Iteration: 5 | log-likelihood: -575.243758066658
## EM - StMoE: Iteration: 6 | log-likelihood: -569.121821835036
## EM - StMoE: Iteration: 7 | log-likelihood: -565.085413323596
## EM - StMoE: Iteration: 8 | log-likelihood: -563.483599829053
## EM - StMoE: Iteration: 9 | log-likelihood: -563.129391506102
## EM - StMoE: Iteration: 10 | log-likelihood: -563.019040796945
## EM - StMoE: Iteration: 11 | log-likelihood: -562.912050661351
## EM - StMoE: Iteration: 12 | log-likelihood: -562.762681987713
## EM - StMoE: Iteration: 13 | log-likelihood: -562.580039526022
## EM - StMoE: Iteration: 14 | log-likelihood: -562.411088197371
## EM - StMoE: Iteration: 15 | log-likelihood: -562.302582513181
## EM - StMoE: Iteration: 16 | log-likelihood: -562.247754786568
## EM - StMoE: Iteration: 17 | log-likelihood: -562.219882633222
## EM - StMoE: Iteration: 18 | log-likelihood: -562.201761352874
## EM - StMoE: Iteration: 19 | log-likelihood: -562.186310255718
## EM - StMoE: Iteration: 20 | log-likelihood: -562.17132753963
## EM - StMoE: Iteration: 21 | log-likelihood: -562.156107460835
## EM - StMoE: Iteration: 22 | log-likelihood: -562.140269866389
## EM - StMoE: Iteration: 23 | log-likelihood: -562.123772098625
## EM - StMoE: Iteration: 24 | log-likelihood: -562.106387456376
## EM - StMoE: Iteration: 25 | log-likelihood: -562.088214763658
## EM - StMoE: Iteration: 26 | log-likelihood: -562.069235287434
## EM - StMoE: Iteration: 27 | log-likelihood: -562.049326951292
## EM - StMoE: Iteration: 28 | log-likelihood: -562.028463524128
## EM - StMoE: Iteration: 29 | log-likelihood: -562.006369869854
## EM - StMoE: Iteration: 30 | log-likelihood: -561.983242167128
## EM - StMoE: Iteration: 31 | log-likelihood: -561.958818064529
## EM - StMoE: Iteration: 32 | log-likelihood: -561.932819250104
## EM - StMoE: Iteration: 33 | log-likelihood: -561.905285699747
## EM - StMoE: Iteration: 34 | log-likelihood: -561.876076243754
## EM - StMoE: Iteration: 35 | log-likelihood: -561.844868244808
## EM - StMoE: Iteration: 36 | log-likelihood: -561.811518798558
## EM - StMoE: Iteration: 37 | log-likelihood: -561.775763365654
## EM - StMoE: Iteration: 38 | log-likelihood: -561.737219240371
## EM - StMoE: Iteration: 39 | log-likelihood: -561.69570438925
## EM - StMoE: Iteration: 40 | log-likelihood: -561.650767022315
## EM - StMoE: Iteration: 41 | log-likelihood: -561.601875555183
## EM - StMoE: Iteration: 42 | log-likelihood: -561.548648250004
## EM - StMoE: Iteration: 43 | log-likelihood: -561.490390688554
## EM - StMoE: Iteration: 44 | log-likelihood: -561.426202767578
## EM - StMoE: Iteration: 45 | log-likelihood: -561.355051717415
## EM - StMoE: Iteration: 46 | log-likelihood: -561.275558122617
## EM - StMoE: Iteration: 47 | log-likelihood: -561.186007492833
## EM - StMoE: Iteration: 48 | log-likelihood: -561.084125214445
## EM - StMoE: Iteration: 49 | log-likelihood: -560.967134271983
## EM - StMoE: Iteration: 50 | log-likelihood: -560.83187601741
## EM - StMoE: Iteration: 51 | log-likelihood: -560.673877228403
## EM - StMoE: Iteration: 52 | log-likelihood: -560.489466300177
## EM - StMoE: Iteration: 53 | log-likelihood: -560.281949348375
## EM - StMoE: Iteration: 54 | log-likelihood: -560.073323321802
## EM - StMoE: Iteration: 55 | log-likelihood: -559.899889267887
## EM - StMoE: Iteration: 56 | log-likelihood: -559.773679074186
## EM - StMoE: Iteration: 57 | log-likelihood: -559.67349172942
## EM - StMoE: Iteration: 58 | log-likelihood: -559.584658109678
## EM - StMoE: Iteration: 59 | log-likelihood: -559.510562092578
## EM - StMoE: Iteration: 60 | log-likelihood: -559.462898651175
## EM - StMoE: Iteration: 61 | log-likelihood: -559.442679098028
## EM - StMoE: Iteration: 62 | log-likelihood: -559.436665069673
## EM - StMoE: Iteration: 63 | log-likelihood: -559.435030878558

Summary

stmoe$summary()
## ------------------------------------------
## Fitted Skew t Mixture-of-Experts model
## ------------------------------------------
## 
## StMoE model with K = 4 experts:
## 
##  log-likelihood df      AIC       BIC       ICL
##        -559.435 30 -589.435 -632.7903 -632.7874
## 
## Clustering table (Number of observations in each expert):
## 
##  1  2  3  4 
## 28 37 31 37 
## 
## Regression coefficients:
## 
##     Beta(k = 1) Beta(k = 2)  Beta(k = 3) Beta(k = 4)
## 1    -3.6985620   992.62382 -1889.618688 292.2075328
## X^1   0.9395230  -103.99905   117.331380 -12.1454859
## X^2  -0.0859998     2.43552    -1.775234   0.1246457
## 
## Variances:
## 
##  Sigma2(k = 1) Sigma2(k = 2) Sigma2(k = 3) Sigma2(k = 4)
##       15.06473      453.8693      491.3541      575.0638

Plots

Mean curve

stmoe$plot(what = "meancurve")

Confidence regions

stmoe$plot(what = "confregions")

Clusters

stmoe$plot(what = "clusters")

Log-likelihood

stmoe$plot(what = "loglikelihood")

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.