The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
Let \[ \hat{e}_{i,j,t}^{[k],h} = y_{i,t+h}^{[k]} - \hat{y}_{i,j,t}^{[k],h}, \quad \begin{array}{l} i=1,\ldots,n, \\ j=0,\ldots,J, \end{array} \; t=1,\ldots, q , \; \begin{array}{l} k \in {\cal K}, \\ h=1,\ldots,h_k , \end{array} \] be the forecast error, where \(y\) and \(\hat{y}\) are the actual and the forecasted values, respectively, suffix \(i\) denotes the variable of interest, \(j\) is the forecasting technique, where \(j=0\) is the benchmark forecasting procedure, \(t\) is the forecast origin, \({\cal K}\) is the set of the time frequencies at which the series is observed, and \(h\) is the forecast horizon, whose lead time depends on the time frequency \(k\).
Denote by A\(_{i,j}^{[k],h}\) the
forecasting accuracy of the technique \(j\), computed across \(q\) forecast origins, for the \(h\)-step-ahead forecasts of the variable
\(i\) at the temporal aggregation level
\(k\). For example, A\(_{i,j}^{[k],h} = MSE_{i,j}^{[k],h}\)
(type=“mse” in score_index()
function,
default), otherwise we might have A\(_{i,j}^{[k],h} = MAE_{i,j}^{[k],h}\)
(type=“mae” in score_index()
function) or A\(_{i,j}^{[k],h} = RMSE_{i,j}^{[k],h}\)
(type=“rmse” in score_index()
function), where
\[
\begin{array}{rcl}
MSE_{i,j}^{[k],h} & = &
\displaystyle\frac{1}{q}\displaystyle\sum_{t=1}^{q}
\left(\hat{e}_{i,j}^{[k],h}\right)^2 \\
MAE_{i,j}^{[k],h} & = &
\displaystyle\frac{1}{q}\displaystyle\sum_{t=1}^{q}
\left|\hat{e}_{i,j}^{[k],h}\right| \\
RMSE_{i,j}^{[k],h} & = &
\sqrt{\displaystyle\frac{1}{q}\displaystyle\sum_{t=1}^{q}
\left(\hat{e}_{i,j}^{[k],h}\right)^2}
\end{array}
\]
In any case, we consider the relative version of the accuracy index \(A_{i,j}^{[k],h}\), given by: \[ r_{i,j}^{[k],h} = \displaystyle\frac{A_{i,j}^{[k],h}}{A_{i,0}^{[k],h}} , \quad i=1,\ldots,n, \quad j=0,\ldots, J, \quad k \in {\cal K}, \quad h=1, \ldots, h_k, \] and use it to compute the Average relative accuracy index of the forecasting procedure \(j\), for given \(k\) and \(h\), through the geometric mean: \[ \text{AvgRelA}_{j}^{[k],h} = \left(\displaystyle\prod_{i=1}^{n} r_{i,j}^{[k],h} \right)^{\frac{1}{n}} , \quad j=0,\ldots,J . \]
We may consider the following average relative accuracy indices for selected groups of variables/time frequencies and forecast horizons:
Average relative accuracy indices for a single variable at a given time frequency, for multiple forecast horizons \[ \label{AvgRelAikh1h2} \text{AvgRelA}_{i,j}^{[k],q_1:q_2} = \left(\prod_{h=q_1}^{q_2} r_{i,j}^{[k],h}\right)^{\frac{1}{q_2 - q_1 + 1}}, \; \begin{array}{l} i=1,\ldots,n, \\ j=0,\ldots,J, \end{array} \; \begin{array}{l} k \in {\cal K}, \\ 1 \le q_1 \le q_2 \le h_k \end{array} . %i=1,\ldots,n , \; j=0,\ldots, J, \; k \in {\cal K}, \; 1 \le q_1 \le q_2 \le h_k. \] Average relative accuracy indices for a group of variables (either all, or selected groups, e.g. a: uts, b: bts) at a given time frequency, either for a single forecast horizon or across them \[ \begin{array}{rcll} \text{AvgRelA}^{[k],h}_j & = & \left(\displaystyle\prod_{i=1}^{n} r_{i,j}^{[k],h}\right)^{\frac{1}{n}}, & j=0,\ldots,J , \; k \in {\cal K}, \; h=1,\ldots,h_k \\[.5cm] \text{AvgRelA}^{[k],h}_{a,j} & = & \left(\displaystyle\prod_{i=1}^{n_a} r_{i,j}^{[k],h}\right)^{\frac{1}{n_a}}, & j=0,\ldots, J, \; k \in {\cal K} \\[.15cm] \text{AvgRelA}^{[k],h}_{b,j} & = & \left(\displaystyle\prod_{i=n_a+1}^{n} r_{i,j}^{[k],h}\right)^{\frac{1}{n_b}}, & j=0,\ldots, J, \; k \in {\cal K} \\[.15cm] \text{AvgRelA}^{[k]}_j & = & \left(\displaystyle\prod_{i=1}^{n} \prod_{h=1}^{h_k} r_{i,j}^{[k],h}\right)^{\frac{1}{n h_k}}, & \; j=0,\ldots,J, \; k \in {\cal K} \\[.15cm] \text{AvgRelA}^{[k]}_{a,j} & = &\left(\displaystyle\prod_{i=1}^{n_a} \prod_{h=1}^{h_k} r_{i,j}^{[k],h}\right)^{\frac{1}{n_a h_k}}, & j=0,\ldots, J, \; k \in {\cal K} \\[.15cm] \text{AvgRelA}^{[k]}_{b,j} & = &\left(\displaystyle\prod_{i=n_a+1}^{n} \prod_{h=1}^{h_k} r_{i,j}^{[k],h}\right)^{\frac{1}{n_b h_k}}, & j=0,\ldots, J, \; k \in {\cal K} \end{array} \]
Average relative accuracy indices for a single variable or for a group of variables (all, a: uts, b: bts), across all time frequencies and forecast horizons \[ \begin{array}{rcll} \text{AvgRelA}_{i,j} & = & \left(\displaystyle\prod_{k \in {\cal K}} \prod_{h=1}^{h_k} r_{i,j}^{[k],h}\right)^{\frac{1}{k^*+m}}, & \begin{array}{l} i=1,\ldots,n \\ j=0,\ldots,J \end{array} \\[.15cm] \text{AvgRelA}_j & = & \left(\displaystyle\prod_{i=1}^{n} \prod_{k \in {\cal K}} \prod_{h=1}^{h_k} r_{i,j}^{[k],h}\right)^{\frac{1}{n(k^*+m)}} , & j=0,\ldots,J \\[.15cm] \text{AvgRelA}_{a,j} & = & \left(\displaystyle\prod_{i=1}^{n_a} \prod_{k \in {\cal K}} \prod_{h=1}^{h_k} r_{i,j}^{[k],h}\right)^{\frac{1}{n_a(k^*+m)}} , & j=0,\ldots,J \\[.15cm] \text{AvgRelA}_{b,j} & = & \left(\displaystyle\prod_{i=n_a+1}^{n} \prod_{k \in {\cal K}} \prod_{h=1}^{h_k} r_{i,j}^{[k],h}\right)^{\frac{1}{n_b(k^*+m)}} , & j=0,\ldots,J \end{array} \]
score_index()
function outputsThe score_index()
function returns a summary table
called Avg\(\_\)mat
(if compact option is TRUE, default),
otherwise it returns a list of four tables. Fixed method \(j\), we have:
\(\textbf{all}\) | \(\textbf{uts}\) | \(\textbf{bts}\) | |
\(\textbf{m}\) | \(\text{AvgRelA}^{[m]}_j\) | \(\text{AvgRelA}^{[m]}_{a,j}\) | \(\text{AvgRelA}^{[m]}_{b,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) |
\(\textbf{k}\) | \(\text{AvgRelA}^{[k]}_j\) | \(\text{AvgRelA}^{[k]}_{a,j}\) | \(\text{AvgRelA}^{[k]}_{b,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) |
\(\textbf{1}\) | \(\text{AvgRelA}^{[1]}_j\) | \(\text{AvgRelA}^{[1]}_{a,j}\) | \(\text{AvgRelA}^{[1]}_{b,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) |
\(\textbf{all}\) | \(\text{AvgRelA}_j\) | \(\text{AvgRelA}_{a,j}\) | \(\text{AvgRelA}_{b,j}\) |
\({\cal K}\) | \(\textbf{m}\) | \(\dots\) | \(\textbf{k}\) | \(\dots\) | \(\textbf{1}\) | ||||
\(\textbf{h}\) | \(\textbf{1}\) | \(\dots\) | \(\textbf{1}\) | \(\dots\) | \(\mathbf{h_k}\) | \(\dots\) | \(\textbf{1}\) | \(\dots\) | \(\mathbf{m}\) |
\(\textbf{1}\) | \(\text{RelA}^{[m],1}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[k],1}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[k],h_k}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[1],1}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[1],m}_{1,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | ||||
\(\textbf{i}\) | \(\text{RelA}^{[m],1}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[k],1}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[k],h_k}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[1],1}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[1],m}_{i,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | ||||
\(\textbf{n}\) | \(\text{RelA}^{[m],1}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[k],1}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[k],h_k}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[1],1}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[1],m}_{n,j}\) |
\({\cal K}\) | \(\textbf{m}\) | \(\dots\) | \(\textbf{k}\) | \(\dots\) | \(\textbf{1}\) | ||||
\(\textbf{h}\) | \(\textbf{1:1}\) | \(\dots\) | \(\textbf{1:1}\) | \(\dots\) | \(\mathbf{1:h_k}\) | \(\dots\) | \(\textbf{1:1}\) | \(\dots\) | \(\mathbf{1:m}\) |
\(\textbf{1}\) | \(\text{RelA}^{[m],1:1}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[k],1:1}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[k],1:h_k}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[1],1:1}_{1,j}\) | \(\dots\) | \(\text{RelA}^{[1],1:m}_{1,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | ||||
\(\textbf{i}\) | \(\text{RelA}^{[m],1:1}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[k],1:1}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[k],1:h_k}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[1],1:1}_{i,j}\) | \(\dots\) | \(\text{RelA}^{[1],1:m}_{i,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | ||||
\(\textbf{n}\) | \(\text{RelA}^{[m],1:1}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[k],1:1}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[k],1:h_k}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[1],1:1}_{n,j}\) | \(\dots\) | \(\text{RelA}^{[1],1:m}_{n,j}\) |
\(\textbf{m}\) | \(\dots\) | \(\textbf{k}\) | \(\dots\) | \(\textbf{1}\) | \(\dots\) | \(\mathbf{all}\) | |
\(\textbf{1}\) | \(\text{AvgRelA}^{[m]}_{1,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k]}_{1,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1]}_{1,j}\) | \(\dots\) | \(\text{AvgRelA}_{1,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | |||
\(\textbf{i}\) | \(\text{AvgRelA}^{[m]}_{i,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k]}_{i,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1]}_{i,j}\) | \(\dots\) | \(\text{AvgRelA}_{i,j}\) |
\(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | \(\vdots\) | |||
\(\textbf{n}\) | \(\text{AvgRelA}^{[m]}_{n,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k]}_{n,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1]}_{n,j}\) | \(\dots\) | \(\text{AvgRelA}_{n,j}\) |
\({\cal K}\) | \(\textbf{m}\) | \(\dots\) | \(\textbf{k}\) | \(\dots\) | \(\textbf{1}\) | ||||
\(\textbf{h}\) | \(\textbf{1}\) | \(\dots\) | \(\textbf{1}\) | \(\dots\) | \(\mathbf{h_k}\) | \(\dots\) | \(\textbf{1}\) | \(\dots\) | \(\mathbf{m}\) |
\(\textbf{all}\) | \(\text{AvgRelA}^{[m],1}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],h_k}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],m}_{j}\) |
\(\textbf{a}\) | \(\text{AvgRelA}^{[m],1}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],h_k}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],m}_{a,j}\) |
\(\textbf{b}\) | \(\text{AvgRelA}^{[m],1}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],h_k}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],m}_{b,j}\) |
\({\cal K}\) | \(\textbf{m}\) | \(\dots\) | \(\textbf{k}\) | \(\dots\) | \(\textbf{1}\) | ||||
\(\textbf{h}\) | \(\textbf{1:1}\) | \(\dots\) | \(\textbf{1:1}\) | \(\dots\) | \(\mathbf{1:h_k}\) | \(\dots\) | \(\textbf{1:1}\) | \(\dots\) | \(\mathbf{1:m}\) |
\(\textbf{all}\) | \(\text{AvgRelA}^{[m],1:1}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1:1}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1:h_k}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1:1}_{j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1:m}_{j}\) |
\(\textbf{a}\) | \(\text{AvgRelA}^{[m],1:1}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1:1}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1:h_k}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1:1}_{a,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1:m}_{a,j}\) |
\(\textbf{b}\) | \(\text{AvgRelA}^{[m],1:1}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1:1}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[k],1:h_k}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1:1}_{b,j}\) | \(\dots\) | \(\text{AvgRelA}^{[1],1:m}_{b,j}\) |
library(FoReco)
data(FoReco_data)
# Cross-temporal framework
<- octrec(FoReco_data$base, m = 12, C = FoReco_data$C,
oct_recf comb = "bdshr", res = FoReco_data$res)$recf
<- score_index(recf = oct_recf,
oct_score base = FoReco_data$base,
test = FoReco_data$test, m = 12, nb = 5)
# Cross-sectional framework#'
# monthly base forecasts
<- FoReco2matrix(FoReco_data$base, m = 12)$k1
mbase # monthly test set
<- FoReco2matrix(FoReco_data$test, m = 12)$k1
mtest # monthly residuals
<- FoReco2matrix(FoReco_data$res, m = 12)$k1
mres # monthly reconciled forecasts
<- htsrec(mbase, C = FoReco_data$C, comb = "shr", res = mres)$recf
mrecf # score
<- score_index(recf = mrecf, base = mbase, test = mtest, nb = 5)
hts_score
# Temporal framework
data(FoReco_data)
# top ts base forecasts ([lowest_freq' ... highest_freq']')
<- FoReco_data$base[1, ]
topbase # top ts residuals ([lowest_freq' ... highest_freq']')
<- FoReco_data$res[1, ]
topres # top ts test ([lowest_freq' ... highest_freq']')
<- FoReco_data$test[1, ]
toptest # top ts recf ([lowest_freq' ... highest_freq']')
<- thfrec(topbase, m = 12, comb = "acov", res = topres)$recf
toprecf # score
<- score_index(recf = toprecf, base = topbase, test = toptest, m = 12) thf_score
Di Fonzo, T., Girolimetto, D. (2021), Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives, International Journal of Forecasting, in press.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.