The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
This vignette provides a “jump-start” for users that want to quickly understand how to use entropy-pooling (EP) to construct views on the market distribution. The methodology fits well in a variety of scenarios: stress-testing, macro factors, non-linear payoffs, and more.
To demonstrate entropy-pooling’s firepower, the current vignette uses some of the cases Meucci (2008)1 leaves for the reader in the appendix. In particular, the usage of the following functions are covered in depth:
view_on_mean()
view_on_volatility()
view_on_correlation()
view_on_rank()
view_on_marginal_distribution()
view_on_copula()
Along with the examples, the EuStockMarkets
dataset -
that comes with the installation of R
- is used as a
proxy for “the market”:
library(ffp)
library(ggplot2)
set.seed(321)
# invariance / stationarity
<- diff(log(EuStockMarkets))
x
dim(x)
#> [1] 1859 4
head(x)
#> DAX SMI CAC FTSE
#> [1,] -0.009326550 0.006178360 -0.012658756 0.006770286
#> [2,] -0.004422175 -0.005880448 -0.018740638 -0.004889587
#> [3,] 0.009003794 0.003271184 -0.005779182 0.009027020
#> [4,] -0.001778217 0.001483372 0.008743353 0.005771847
#> [5,] -0.004676712 -0.008933417 -0.005120160 -0.007230164
#> [6,] 0.012427042 0.006737244 0.011714353 0.008517217
Assume an investor believes the FTSE
index return will
be \(20\%\) above average in the near
future. In order to process this view, the investor needs to
minimize the relative entropy:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) \]
Subject to the restriction:
\[ \ x_j V_{j, k} = \mu_{k} \]
In which \(x_{j}\) is a yet to be
discovered posterior probability; \(V_{j,k}\) is a \(1859 \times 1\) vector with the
FTSE
returns; and \(\mu_{k}\) is a \(1 \times 1\) scalar with the investor
projected return for the FTSE
index. In this case, the
\(k\) subscript represents the fourth
column in x
2.
Views on expected returns can be constructed with
view_on_mean
:
# Returns 20% higher than average
<- mean(x[ , "FTSE"]) * 1.2
mu
# ffp views constructor
<- view_on_mean(x = as.matrix(x[ , "FTSE"]), mean = mu)
views
views#> # ffp view
#> Type: View On Mean
#> Aeq : Dim 1 x 1859
#> beq : Dim 1 x 1
The views
object is a list
with two
components, Aeq
and beq
, that are equivalent
to the elements \(V_{j,k}\) transposed
and \(\mu_{k}\), respectively.
The investor also needs to formulate a vector of prior probabilities, \(p_j\), which is usually - but not necessary - a equal-weight vector:
# Prior probabilities
<- rep(1 / nrow(x), nrow(x)) p_j
Once the prior and the views are established, the
optimization can take place with entropy_pooling
:
# Solve Minimum Relative Entropy (MRE)
<- entropy_pooling(
ep p = p_j,
Aeq = views$Aeq,
beq = views$beq,
solver = "nlminb"
)
ep#> <ffp[1859]>
#> 0.0005425631 0.0005340012 0.000544236 0.0005418246 0.0005322989 ... 0.0005451271
What is the interpretation for ep
? Among all the
possible probability vectors, ep
is the one that can
satisfy the views in the “least” evasive way (in which the term
“least evasive” is direct reference to the prior). Therefore,
ep
is the best candidate for a posterior
distribution.
In real world, the investor can have many views. Extending
the current example, say the investor has a new belief: the
CAC
index returns will be \(10\%\) bellow average3:
<- c(
mu mean(x[ , "FTSE"]) * 1.2, # Return 20% higher than average
mean(x[ , "CAC"]) * 0.9 # Return 10% bellow average
)
# ffp views constructor
<- view_on_mean(x = as.matrix(x[ , c("FTSE", "CAC")]), mean = mu)
views
views#> # ffp view
#> Type: View On Mean
#> Aeq : Dim 2 x 1859
#> beq : Dim 2 x 1
In which the elements Aeq
and beq
now have
\(2\) rows each, one for every
view.
The minimum relative entropy problem is solved in the exact same way:
# Solve MRE
<- entropy_pooling(
ep p = p_j,
Aeq = views$Aeq,
beq = views$beq,
solver = "nlminb"
)
ep#> <ffp[1859]>
#> 0.0005602371 0.0005473082 0.0005573006 0.0005384979 0.000531066 ... 0.0005434801
It’s easier to analyse the output of ep
visually.
class(ep)
#> [1] "ffp" "vctrs_vctr"
To that end, the autoplot
method is available for
objects of the ffp
class:
autoplot(ep) +
scale_color_viridis_c(option = "C", end = 0.75) +
labs(title = "Posterior Probability Distribution",
subtitle = "Views on Expected Returns",
x = NULL,
y = NULL)
The plot reinforces the intuition of \(x_j\) - the posterior probability - as a distribution that distorts the prior in order to accommodate the views.
It’s easy to double-check the investor views by computing the ratio of the conditional vs. unconditional returns:
<- ffp_moments(x, ep)$mu
conditional <- colMeans(x)
unconditional
/ unconditional - 1
conditional #> DAX SMI CAC FTSE
#> 0.01335825 0.02078130 -0.09999999 0.20000009
According to expectations, CAC
outlook is now \(10\%\) bellow average and FTSE
expected return is \(20\%\) above
average. Violà!
Say the investor is confident that markets will be followed by a
period of lull and bonanza. Without additional insights about expected
returns, the investor anticipates that market volatility for the
CAC
index will drop by \(10\%\).
To impose views on volatility, the investor needs to minimize the expression:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) \] Subject to the constraint:
\[ \sum_{j=1}^{J} x_j V_{j, k}^2 = \mu_{k}^2 + \sigma_{k}^2 \]
In which, \(x_j\) is a vector of yet
to be defined posterior probabilities; \(V_{j,k}^2\) is a \(1859 \times 1\) vector with
CAC
squared returns; \(\mu_{k}^2\) and \(\sigma_{k}^2\) are the CAC
sample mean and variance; and \(k\) is
the column position of CAC
in the panel \(V\).
The view_on_volatility
function can be used for this
purpose:
# opinion
<- sd(x[ , "CAC"]) * 0.9
vol_cond
# views
<- view_on_volatility(x = as.matrix(x[ , "CAC"]), vol = vol_cond)
views
views#> # ffp view
#> Type: View On Volatility
#> Aeq : Dim 1 x 1859
#> beq : Dim 1 x 1
The views
object holds a list with two components -
Aeq
and beq
- that are equivalent to the
elements \(V_{j,k}^2\) transposed and
\(\mu_{k}^2 + \sigma_{k}^2\),
respectively.
To solve the relative entropy use the entropy_pooling
function:
<- entropy_pooling(
ep p = p_j,
Aeq = views$Aeq,
beq = views$beq,
solver = "nlminb"
)
ep#> <ffp[1859]>
#> 0.0005227495 0.0004701207 0.0005609239 0.0005476659 0.0005631671 ... 0.0005349393
Once again, the ep
vector is what we call
posterior: the probability vector that causes the “least”
amount of distortion in the prior, but still obeys the
constraints (views).
The posterior probabilities can be observed with the
autoplot
method:
autoplot(ep) +
scale_color_viridis_c(option = "C", end = 0.75) +
labs(title = "Posterior Probability Distribution",
subtitle = "View on Volatility",
x = NULL,
y = NULL)
To check if the posterior probabilities are valid as a way to display the investor’s view, compare the conditional vs. unconditional volatilities:
<- apply(x, 2, sd)
unconditional <- sqrt(diag(ffp_moments(x, ep)$sigma))
conditional
/ unconditional - 1
conditional #> DAX SMI CAC FTSE
#> -0.08377407 -0.06701178 -0.09977906 -0.04552477
The posterior volatility for the CAC
index is
reduced by \(10\%\), in line with the
investor’s subjective judgment. However, by emphasizing tranquil periods
more often, other assets are also affected, but in smaller
magnitudes.
Assume the investor believes the correlation between
FTSE
and DAX
will increase by \(30\%\), from \(0.64\) to \(0.83\). To construct views on the
correlation matrix, the general expression has to be minimized:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) \] Subject to the constraints:
\[ \sum_{j=1}^{J} x_j V_{j, k} V_{j, l} = \mu_{k} \mu_{l} + \sigma_{k} \sigma_{l} C_{k,l} \]
In which, the term \(V_{j, k} V_{j, l}\) on the left hand-side of the restriction consists of cross-correlations among assets; the terms \(\mu_{k} \mu_{l} + \sigma_{k} \sigma_{l} C_{k,l}\) carry the investor target correlation structure; and \(x_j\) is a yet to be defined probability vector.
To formulate this view, first compute the unconditional correlation matrix. Second, add a “perturbation” in the corresponding element that is consistent with the perceived view:
<- cor(x) # unconditional correlation matrix
cor_view "FTSE", "DAX"] <- 0.83 # perturbation (investor belief)
cor_view["DAX", "FTSE"] <- 0.83 # perturbation (investor belief) cor_view[
Finally, pass the adjusted correlation matrix into the
view_on_correlation
function:
<- view_on_correlation(x = x, cor = cor_view)
views
views#> # ffp view
#> Type: View On Correlation
#> Aeq : Dim 10 x 1859
#> beq : Dim 10 x 1
In which Aeq
is equal to \(V_{j, k} V_{j, l}\) transposed and
beq
is a \(10 \times 1\)
vector with \(\mu_{k} \mu_{l} + \sigma_{k}
\sigma_{l} C_{k,l}\).
Notice that even though the investor has a view in just one parameter4 the constraint vector has \(10\) rows, because every element of the lower/upper diagonal of the correlation matrix has to match.
Once again, the minimum entropy is solved by
entropy_pooling
:
<- entropy_pooling(
ep p = p_j,
Aeq = views$Aeq,
beq = views$beq,
solver = "nlminb"
)
ep#> <ffp[1859]>
#> 6.709143e-05 0.000659403 0.001085763 0.0003254822 0.0005215729 ... 0.0004748052
And the autoplot
method is available:
autoplot(ep) +
scale_color_viridis_c(option = "C", end = 0.75) +
labs(title = "Posterior Probability Distribution",
subtitle = "View on Correlation",
x = NULL,
y = NULL)
To fact-check the investor view, compute the posterior correlation matrix:
cov2cor(ffp_moments(x, p = ep)$sigma)
#> DAX SMI CAC FTSE
#> DAX 1.0000000 0.6932767 0.7317714 0.8346939
#> SMI 0.6932767 1.0000000 0.6260098 0.6113109
#> CAC 0.7317714 0.6260098 1.0000000 0.6535585
#> FTSE 0.8346939 0.6113109 0.6535585 1.0000000
And notice that the linear association between FTSE
and
DAX
is now \(0.83\)!
Assume the investor believes the DAX
index will
outperform the SMI
by some amount, but he doesn’t know by
how much. If the investor has only a mild view on the
performance of assets, he may want to minimize the following
expression:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) \] Subject to the constraint:
\[ \sum_{j=1}^{J} x_j (V_{j, k} - V_{j,
l}) \ge 0 \] In which, the \(x_j\) is a yet to be determined probability
vector; \(V_{j, k}\) is a \(1859 \times 1\) column vector with
DAX
returns and \(V_{j,
l}\) is a \(1859 \times 1\)
column vector with the SMI
returns. In this case, the \(k\) and \(l\) subscripts refers to the column
positions of DAX
and SMI
in the object
x
, respectively.
Views on relative performance can be imposed with
view_on_rank
:
<- view_on_rank(x = x, rank = c(2, 1))
views
views#> # ffp view
#> Type: View On Rank
#> A : Dim 1 x 1859
#> b : Dim 1 x 1
Assets that are expected to outperform enter in the rank
argument with their column positions to the right: assets in the left
underperform and assets in the right outperform. Because the investor
believes that DAX
\(\geq\)
SMI
(the asset in the first column will outperform the
asset in the second column), we fill rank = c(2, 1)
5.
The optimization is, once again, guided by the
entropy_pooling
function:
<- entropy_pooling(
ep p = p_j,
A = views$A,
b = views$b,
solver = "nloptr"
)
ep#> <ffp[1859]>
#> 0.0005145646 0.0005403155 0.0005470049 0.0005330238 0.0005446858 ... 0.0005469164
Two important observations:
nlminb
solver. Use nloptr
or
solnl
, instead;A
and
b
to be fulfilled rather than Aeq
and
beq
.The posterior probabilities are presented bellow:
autoplot(ep) +
scale_color_viridis_c(option = "C", end = 0.75) +
labs(title = "Posterior Probability Distribution",
subtitle = "View on Ranking/Momentum",
x = NULL,
y = NULL)
To fact-check the ongoing view, compare the conditional vs. unconditional returns:
<- ffp_moments(x, ep)$mu
conditional <- colMeans(x)
unconditional
/ unconditional - 1
conditional #> DAX SMI CAC FTSE
#> 0.17274275 -0.06507207 0.13576463 0.06261163
Indeed, returns for DAX
are adjusted upwards by \(17\%\), while returns for SMI
are revised downwards by \(6.5\%\).
Notice that returns for CAC
and FTSE
were also
affected. This behavior could be tamed by combining the ranking
condition with views on expected returns. See the function
bind_views()
for an example.
Assume the investor has a view on the entire marginal distribution. One way to add views on marginal distributions is by matching the first moments exactly, up to a given order. Mathematically, the investor minimizes the relative entropy:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) \] With, say, two equality constraints (one for \(\mu\) and one for \(\sigma^2\)):
\[ \sum_{j=1}^J x_j V_{j,k} = \sum_{j=z}^Z p_z \hat{V}_{z,k} \] \[ \sum_{j=1}^J x_j (V_{j,k})^2 = \sum_{z=1}^Z p_z (\hat{V}_{z,k})^2 \]
In which \(x_j\) is a yet to be defined probability vector; \(V_{j,k}\) is a matrix with the empirical marginal distributions; \(p_z\) is vector of prior probabilities; and \(\hat{V}_{z,k}\) an exogenous panel with simulations that are consistent with the investor’s views.
When \(j = z\), the panels \(V_{j,k}\) and \(\hat{V}_{z,k}\) have the same number of rows and the dimensions in both sides of the restrictions match. However, it’s possible to set \(z \ge j\) to simulate a larger panel for \(\hat{V}_{z, k}\). Keep in mind though that, if \(z \ne j\), two prior probabilities will have to be specified: one for \(p_j\) (the objective function) and one for \(p_z\) (the views).
Continuing on the example, consider the margins of x
can
be approximated by a symmetric multivariate t-distribution. If this is
the case, the estimation can be conducted by the amazing
ghyp
package, that covers the entire family of generalized
hyperbolic distributions:
library(ghyp)
# multivariate t distribution
<- fit.tmv(data = x, symmetric = TRUE, silent = TRUE)
dist_fit
dist_fit#> Symmetric Student-t Distribution:
#>
#> Parameters:
#> nu
#> 6.151241
#>
#> mu:
#> [1] 0.0007899665 0.0009594760 0.0004790250 0.0003811669
#>
#> sigma:
#> DAX SMI CAC FTSE
#> DAX 1.000013e-04 6.046969e-05 7.933384e-05 5.072439e-05
#> SMI 6.046969e-05 8.062728e-05 5.869208e-05 4.119624e-05
#> CAC 7.933384e-05 5.869208e-05 1.216927e-04 5.715865e-05
#> FTSE 5.072439e-05 4.119624e-05 5.715865e-05 6.398099e-05
#>
#> gamma:
#> [1] 0 0 0 0
#>
#> log-likelihood:
#> 26370.73
#>
#>
#> Call:
#> fit.tmv(data = x, symmetric = TRUE, silent = TRUE)
The investor then, can construct a large panel with statistical properties similar to the margins he envisions:
set.seed(123)
# random numbers from `dist_fit`
<- rghyp(n = 100000, object = dist_fit) simul
Where the \(100.000 \times 4\)
matrix is used to match the views with
view_on_marginal_distribution
:
<- rep(1 / 100000, 100000)
p_z <- view_on_marginal_distribution(
views x = x,
simul = simul,
p = p_z
)
views#> # ffp view
#> Type: View On Marginal Distribution
#> Aeq : Dim 8 x 1859
#> beq : Dim 8 x 1
The objects simul
and p_z
corresponds to
the terms \(\hat{V}_{z,k}\) transposed
and \(p_z\), respectively. Note that
\(8\) restrictions are created, four
for each moment of the marginal distribution (there are four assets in
“the market”).
With the prior and the views at hand, the optimization can be quickly implemented:
<- entropy_pooling(
ep p = p_j,
Aeq = views$Aeq,
beq = views$beq,
solver = "nloptr"
)
ep#> <ffp[1859]>
#> 0.000528748 0.0005678894 0.000536657 0.0005231905 0.0005349765 ... 0.0005324979
As the visual inspection of the vector ep
:
autoplot(ep) +
scale_color_viridis_c(option = "C", end = 0.75) +
labs(title = "Posterior Probability Distribution",
subtitle = "View on Marginal Distribution",
x = NULL,
y = NULL)
It’s easy to extend the current example. For instance, the investor could “tweak” some of the fitted parameters for stress-testing (i.e. change the degrees of freedom, increase/decrease expected returns, etc) to verify the immediate impact on the P&L (VaR, CVaR, etc).
Acknowledge that when the restrictions are to harsh, the scenarios
will be heavily distorted to satisfy the views. In this case,
one may need to compute the effective number of scenarios (ENS). This is
very easy to do with ens()
:
ens(ep)
#> [1] 1857.356
Rest on the investor to calibrate this statistic in order to get a desired confidence level6.
Assume the investor would like to simulate a different dependence structure for the market. Views that change the interdependence of assets can be implemented by minimizing the relative entropy:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j)) \] Subject to the following restrictions:
\[ \sum_{j=1}^J x_i U_{j,k} = 0.5\] \[ \sum_{j=1}^J x_j U_{j,k}U_{j,l} = \sum_{z=1}^Z p_z \hat{U}_{z,k}\hat{U}_{z,l} \]
\[ \sum_{j=1}^J x_j U_{j,k}U_{j,l}U_{j,i} = \sum_{j=1}^J p_j \hat{U}_{j,k}\hat{U}_{j,l}\hat{U}_{j,i} \]
In which, the first restriction matches the first moment of the uniform distribution; the second and third restrictions pair the cross-moments of the empirical copula, \(U\), with the simulated copula, \(\hat{U}\); \(x_j\) is a yet to be discovered posterior distribution; \(p_z\) is a prior probability; When \(j = z\), the dimensions of \(p_j\) and \(p_z\) match.
Among many of the available copulas, say the investor wants to model
the dependence of the market as a clayton
copula to ensure the lower tail dependency
does not go unnoticed. The estimation is simple to implement with the
package copula
:
library(copula)
# copula (pseudo-observation)
<- pobs(x)
u
# copula estimation
<- fitCopula(
cop copula = claytonCopula(dim = ncol(x)),
data = u,
method = "ml"
)
# simulation
<- rCopula(
r_cop n = 100000,
copula = claytonCopula(param = cop@estimate, dim = 4)
)
First, the copulas are computed component-wise by applying the
marginal empirical distribution in every \(k\) column of the dataset. Second, the
pseudo-observations are used to fit a Clayton copula by the
Maximum-Likelihood (ML) method. Finally, a large panel with dimension
\(100.000 \times 4\) is constructed to
match a Clayton copula with \(\hat \alpha =
1.06\), as in the object cop
.
The views on the market is constructed with the
view_on_copula
:
<- view_on_copula(x = u, simul = r_cop, p = p_z)
views
views#> # ffp view
#> Type: View On Copula
#> Aeq : Dim 34 x 1859
#> beq : Dim 34 x 1
Once again, the solution of the minimum relative entropy can be found in one line of code:
<- entropy_pooling(
ep p = p_j,
Aeq = views$Aeq,
beq = views$beq,
solver = "nloptr"
)
ep#> <ffp[1859]>
#> 0.0001990935 0.0004161479 0.0008166059 0.0006512624 0.0003776943 ... 0.0003269579
And the autoplot
method is available for the objects of
the ffp
class:
autoplot(ep) +
scale_color_viridis_c(option = "C", end = 0.75) +
labs(title = "Posterior Probability Distribution",
subtitle = "View on Copulas",
x = NULL,
y = NULL)
To stretch the current example, assume the investor stresses the parameter \(\hat \alpha = 1.06\)7. Higher \(\alpha\) values are linked to extreme occurrences on the left tails of the distribution. Whence, consider the case where \(\hat \alpha = 5\):
@estimate <- 5 cop
Rerun the previous steps (simulation, views and optimization):
# simulation
<- rCopula(
r_cop n = 100000,
copula = claytonCopula(param = cop@estimate, dim = 4)
)
# #views
<- view_on_copula(x = u, simul = r_cop, p = p_z)
views
# relative entropy
<- entropy_pooling(
ep p = p_j,
Aeq = views$Aeq,
beq = views$beq,
solver = "nloptr"
)
To find the new posterior probability vector the satisfy the stress-test condition:
autoplot(ep) +
scale_color_viridis_c(option = "C", end = 0.75) +
labs(title = "Posterior Probability Distribution",
subtitle = "View on Copulas",
x = NULL,
y = NULL)
Meucci, Attilio, Fully Flexible Views: Theory and Practice (August 8, 2008). Fully Flexible Views: Theory and Practice, Risk, Vol. 21, No. 10, pp. 97-102, October 2008, Available at SSRN: https://www.ssrn.com/abstract=1213325↩︎
The commands x[ , 4]
or
x[ , "FTSE"]
yield the same results.↩︎
An ARIMA, GARCH or VAR model could be used as well. The focus is not on the best forecasting method, but how to input the view once the forecast has already been made.↩︎
cor(ftse, dax) = 0.83
.↩︎
In the x
object, DAX
it’s in
the first column and the SMI
it’s in the second column.↩︎
Meucci, Attilio, Effective Number of Scenarios in Fully Flexible Probabilities (January 30, 2012). GARP Risk Professional, pp. 32-35, February 2012, Available at SSRN: https://www.ssrn.com/abstract=1971808.↩︎
cop@estimate
↩︎
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.