Summary
The R-package qad (short for quantification of asymmetric dependence)
allows to quantify/estimate the (directed) dependence of two variables
\(X\) and \(Y\). The implemented estimator is copula
based, hence scale-invariant, and estimates a directed (population
based) dependence measure \(q(X,Y)\)
attaining values in \([0,1]\) and
having the following main property: \(q(X,Y)\) is \(1\) if and only if \(Y\) is a function of \(X\) (knowing \(X\) means knowing \(Y\)) and \(0\) if and only if \(X\) and \(Y\) are independent (no information gain).
While the Pearson correlation coefficient assesses only linear and
Spearman rank correlation only monotonic relationships, qad is able to
detect any kind of association.
Motivation
All statistics courses mention independence. Loosely speaking, two
random variables \(X\) and \(Y\) are called independent, if \(X\) has no influence on \(Y\) AND (by definition) vice versa.
Example (Rolling a dice twice - independence):
Let’s assume that \(X\) and \(Y\) are random variables, whereby
- \(X\) is the result of rolling a
dice the first time and
- \(Y\) is the result of rolling the
same dice a second time.
If we know \(X\) (the outcome of
rolling the dice the first time), does it help to predict \(Y\)? Obviously not. The probabilities/the
distribution of \(Y\) remain(s)
unchanged, i.e., we do not gain any knowledge about \(Y\) if we know \(X\) (in fact, \(Y\) is still uniformly distributed on \(\{1,\ldots,6\}\)) and vice versa. In other
words: Knowing \(X\) does not reduce
the uncertainty about \(Y\) and vice
versa (try out the R-Code).
R <- 100000
X <- sample(1:6, R, replace = T)
Y <- sample(1:6, R, replace = T)
#Probability that Y = 1
mean(ifelse(Y == 1, 1, 0))
#Probability that Y = 1 if we know X
probs <- rep(0,6)
for(i in 1:6){
probs[i] <- sum(ifelse(Y == 1 & X == i, 1, 0))/sum(X == i)
}
#Probability that Y = 1 if we know the value of X
probs
What is the opposite of independence? Consider the following
example:
Example (Almost complete dependence)
Suppose that \((x_1,y_1), \ldots, (x_n,
y_n)\) is a sample of size \(n=40\) from the model \(Y=X^2+\varepsilon\) with noise \(\varepsilon \sim \mathcal{N}(0,0.05)\) (see
Figure):
#> Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
#> ℹ Please use `linewidth` instead.
Which variable is easier to predict given the value of the other one?
Obviously, knowing the value of \(X\)
provides more information about the value of \(Y\) than vice versa. Indeed, from
the (estimated) equation \(Y=X^2\) we
can determine the value of Y, in the other direction, however, we have
two possibilities for X given Y.
One natural question arising both theoretically as well as in
practise is whether it is possible to quantify and estimate the extent
of dependence of two random variables in full generality, i.e., without
any distributional assumptions.
Commonly used measures of association such as Pearson \(r\) or Spearman \(\rho\) fail to detect any dependence for
the situation depicted in the previous Figure. Furthermore,
interchanging \(X\) and \(Y\) yields the same result, and we get
- \(r(X,Y)=r(Y,X)=\) 0.02
- \(\rho(X,Y)=\rho(Y,X)=\)
-0.112
Long story short: methods other than standard correlation are
needed.
qad
qad (short for quantification of asymmetric
dependence) is a strongly consistent estimator \(q_n(X,Y)\) of the copula-based, hence
scale-invariant directed dependence measure \(q(X,Y)\) (originally called \(\zeta_1\)) introduced in
Trutschnig,
2011. The qad estimator \(q_n(X,Y)\) was developed and analyzed in
Junker,
Griessenberger, Trutschnig, 2021 and implemented in the R-package
qad in 2020. The population value \(q(X,Y)\) has the following key
properties:
- \(q(X,Y)\) can be calculated for
all (continuous) random variables \(X\)
and \(Y\) (without any parametric
knowledge about the distribution/model)
- \(q(X,Y) \in [0,1]\)
(normalization)
- \(q(X,Y) = 0\) if and only if \(X\) and \(Y\) are independent (independence)
- \(q(X,Y) = 1\) if and only if \(Y\) is a function of \(X\) (but not necessarily vice versa), i.e.,
if we have \(Y=f(X)\), so if we are in
the situation that knowing \(X\) means
knowing \(Y\) exactly (think of the
Example without noise); this situation is usually referred to as
complete dependence or full predictability
- We do not necessarily have \(q(X,Y) =
q(Y,X)\) (asymmetry)
- Scale changes do not affect \(q(X,Y)\) (scale-invariance)
Applying qad to the sample depicted in the previous Figure
yields \(q_n(X,Y)=\) 0.778, indicating
a strong influence of \(X\) on \(Y\). On the other hand, \(q_n(Y,X) =\) 0.439, i.e., the estimated
influence of \(Y\) on \(X\) is much lower. In other words: \(Y\) is better predictable by \(X\) than vice versa, hence the quantity
\(a\) denoting asymmetry in dependence
is positive: \(a_n:=q_n(X,Y)-q_n(Y,X)=\) 0.34.
In what follows the qad approach is sketched and some code examples
are presented. We refer to
Junker,
Griessenberger, Trutschnig, 2021 and
Trutschnig,
2011 for a concise mathematical introduction and theoretical
results.
The idea and the approach behind qad
The qad estimator \(q_n(X,Y)\) is
strongly consistent in full generality, i.e., we have \(q_n(X,Y) \approx q(X,Y)\) for sufficiently
large \(n\) (see
Junker,
Griessenberger, Trutschnig, 2021). The underlying estimation
procedure can be sketched as follows:
- (S0) Suppose that \((x_1,
y_1),\ldots , (x_n, y_n)\) is a sample from \((X,Y)\).
- (S1) Calculate the normalized ranks of the sample (we get
values of the form \((i/n, j/n)\) with
\(i, j \in \{1, . . . , n\}\)) as well
as the so-called empirical copula \(\hat{E}_n\).
- (S2) Aggregate the empirical copula to the empirical
checkerboard copula \(CB_N(\hat{E}_n)\)
(a.k.a. 2-dimensional histogram in the copula setting). The masses of
the little squares are aggregated/summed up to the larger \(N \times N\) squares; the resolution N
depends on the sample size n, in the current case we have \(N=6\).
- (S3) Calculate how different the checkerboard distribution
and the uniform distribution on the unit square (modelling independence)
are. More precisely, the conditional distribution functions of the
checkerboard copula are compared with the distribution function of the
uniform distribution on \([0,1]\) (in
the sense that the area between the graphcs is
calculated).
- (S4) Computing the sum of all areas and normalizing
appropriately yields \(q_n(X,Y)=\)
0.778 as well as \(q_n(Y,X) =\)
0.439.
Some examples
Example 1 (Complete dependence):
The following simulated example illustrates again how qad is capable
of picking up asymmetry in dependence where standard measures fail. We
generate a sample of size \(n=100\)
drawn from a sinusoidal association (a lof of information gain about
\(Y\) by knowing \(X\), less vice versa) and compute qad in
both directions using the function qad().
set.seed(1)
## Step 1: Generate sample
n <- 100
#Underlying Model Y = sin(X) + small.error
X <- runif(n, -10, 10)
Y <- sin(X) + rnorm(n, 0, 0.1)
#Plot the sample
plot(X,Y, pch = 16)
#Compute the dependence measure q_n (and the additional p-values obtained by testing for q=0 and a=0)
fit <- qad(X,Y, p.value = T)
#>
#> quantification of asymmetric dependence:
#>
#> Data: x1 := X
#> x2 := Y
#>
#> Sample Size: 100
#> Number of unique ranks: x1: 100
#> x2: 100
#> (x1,x2): 100
#> Resolution: 10 x 10
#>
#> Dependence measures:
#> q p.values
#> q(x1,x2) 0.678 0.000
#> q(x2,x1) 0.309 0.099
#> max.dependence 0.678 0.000
#>
#> a p.values
#> asymmetry 0.369 NA
According to \(q(x_1,x_2) >
q(x_2,x_1)\) (in the notation from before meaning that \(q_n(X,Y)>q_n(Y,X)\)) the qad estimator
informs us that \(X\) provides more
information about \(Y\) than vice
versa. The qad function additionally calculates the maximum dependence,
i.e., \(\max\{q(x_1,x_2),q(x_2,x_1)\}\), as well as
the asymmetry value \(a=q(x_1,x_2)-q(x_2,x_1)\). Moreover, the
output of qad provides p.values obtained by testing for independence via
resampling methods (hypotheses are rejected at the standard significance
level).
Example 2 (Independence):
The following simulated example shows that qad also detects
independence. In fact, generating an independent sample of size \(n=100\) and computing qad in both
directions using the function qad() yields the following:
set.seed(1)
## Step 1: Generate sample
n <- 100
#Underlying Model Y = sin(X) + error
X <- rnorm(n, 0, 10)
Y <- rnorm(n, 0, 20)
#Plot the sample
plot(X,Y, pch = 16)
#Compute the dependence measure q
fit <- qad(X,Y, p.value = T)
#>
#> quantification of asymmetric dependence:
#>
#> Data: x1 := X
#> x2 := Y
#>
#> Sample Size: 100
#> Number of unique ranks: x1: 100
#> x2: 100
#> (x1,x2): 100
#> Resolution: 10 x 10
#>
#> Dependence measures:
#> q p.values
#> q(x1,x2) 0.237 0.697
#> q(x2,x1) 0.273 0.334
#> max.dependence 0.273 0.544
#>
#> a p.values
#> asymmetry -0.037 NA
Although the q-values are greater than \(0\) (which is always the case by
construction - the areas in (S3) can not be smaller than \(0\)), the p.values corresponding to \(q(x_1,x_2)\) and \(q(x_2,x_1)\) are 0.697 and 0.334,
respectively. Both are much greater than \(0.05\), so the null hypothesis of
independence of the underlying random variables \(X\) and \(Y\) is NOT rejected. Notice that for small
sample sizes the q-values can be large, nevertheless according to the
p.value the null hypothesis of independence is rejected (run the above
code with \(n=20\)).
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
