Partial correlation

In probability theory and statistics, partial correlation measures the degree of association between two random variables, with the effect of a set of controlling random variables removed.

Formal definition
Formally, the partial correlation between X and Y given a set of n controlling variables Z = {Z1, Z2, …, Zn}, written &rho;XY·Z, is the correlation between the residuals RX and RY resulting from the linear regression of X with Z and of Y with Z, respectively.

Using linear regression
The obvious way to compute a (sample) partial correlation is to solve the two associated linear regression problems, get the residuals, and calculate the correlation between the residuals. If we write xi, yi and zi to denote i.i.d. samples of some joint probability distribution over X, Y and Z, solving the linear regression problem amounts to finding


 * $$\mathbf{w}_X^* = \arg\min_{\mathbf{w}} \left\{ \sum_{i=1}^N (x_i - \langle\mathbf{w}, \mathbf{z}_i \rangle)^2 \right\} $$
 * $$\mathbf{w}_Y^* = \arg\min_{\mathbf{w}} \left\{ \sum_{i=1}^N (y_i - \langle\mathbf{w}, \mathbf{z}_i \rangle)^2 \right\} $$

with N being the number of samples and $$\langle\mathbf{v},\mathbf{w} \rangle$$ the scalar product between the vectors v and w. The residuals are then


 * $$r_{X,i} = x_i - \langle\mathbf{w}_X^*,\mathbf{z}_i \rangle$$
 * $$r_{Y,i} = y_i - \langle\mathbf{w}_Y^*,\mathbf{z}_i \rangle$$

and the sample partial correlation is


 * $$\hat{\rho}_{XY\cdot\mathbf{Z}}=\frac{N\sum_{i=1}^N r_{X,i}r_{Y,i}-\sum_{i=1}^N r_{X,i}\sum r_{Y,i}}

{\sqrt{N\sum_{i=1}^N r_{X,i}^2-\left(\sum_{i=1}^N r_{X,i}\right)^2}~\sqrt{N\sum_{i=1}^N r_{Y,i}^2-\left(\sum_{i=1}^N r_{Y,i}\right)^2}}.$$

Using recursive formula
It can be computationally expensive to solve the linear regression problems. Actually, the nth-order partial correlation (i.e., with |Z| = n) can be easily computed from three (n - 1)th-order partial correlations. The zeroth-order partial correlation &rho;XY·Ø is defined to be the regular correlation coefficient &rho;XY.

It holds, for any $$Z_0 \in \mathbf{Z}$$:
 * $$\rho_{XY\cdot \mathbf{Z} } =

\frac{\rho_{XY\cdot\mathbf{Z}\setminus\{Z_0\}} - \rho_{XZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}\rho_{YZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}} {\sqrt{1-\rho_{XZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}^2} \sqrt{1-\rho_{YZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}^2}}.$$

Naïvely implementing this computation as a recursive algorithm yields an exponential time complexity. However, this computation has the overlapping subproblems property, such that using dynamic programming or simply caching the results of the recursive calls yields a complexity of $$\mathcal{O}(n^3)$$.

Using matrix inversion
Another approach allows to compute in $$\mathcal{O}(n^3)$$ all partial correlations between any two variables Xi and Xj of a set V of cardinality n given all others, i.e., $$\mathbf{V} \setminus \{X_i,X_j\}$$, provided the correlation matrix Ω = (ωij), where ωij = ρX iXj , is invertible. If we define P = Ω-1, we have:


 * $$\rho_{X_iX_j\cdot \mathbf{V} \setminus \{X_i,X_j\}} = -\frac{p_{ij}}{\sqrt{p_{ii}p_{jj}}}.$$

Geometrical
Let three variables X, Y, Z be chosen from a joint probability distribution over n variables V. Further let vi, 1 ≤ i ≤ N, be N n-dimensional i.i.d. samples taken from the joint probability distribution over V. We then consider the N-dimensional vectors x (formed by the successive values of X over the samples), y (formed by the values of Y) and z (formed by the values of Z).

It can be shown that the residuals RX coming from the linear regression of X using Z, if also considered as an N-dimensional vector rX, have a zero scalar product with the vector z generated by Z. This means that the residuals vector lives on a hyperplane Sz which is perpendicular to z.

The same also applies to the residuals RY generating a vector rY. The desired partial correlation is then the cosine of the angle φ between the projections rX and rY of x and y, respectively, onto the hyperplane perpendicular to z.

As conditional independence test
With the assumption that all involved variables are multivariate Gaussian, the partial correlation &rho;XY·Z is zero if and only if X is conditionally independent from Y given Z. This property does not hold in the general case.

In order to test if a sample partial correlation $$\hat{\rho}_{XY\cdot\mathbf{Z}}$$ vanishes, Fisher's z-transform of the partial correlation can be used:


 * $$z(\hat{\rho}_{XY\cdot\mathbf{Z}}) = \frac{1}{2} \ln\left(\frac{1+\hat{\rho}_{XY\cdot\mathbf{Z}}}{1-\hat{\rho}_{XY\cdot\mathbf{Z}}}\right).$$

The null hypothesis is $$H_0: \hat{\rho}_{XY\cdot\mathbf{Z}} = 0$$, to be tested against the two-tail alternative $$H_A: \hat{\rho}_{XY\cdot\mathbf{Z}} \neq 0$$. We reject H0 with significance level α if:


 * $$\sqrt{N - |\mathbf{Z}| - 3}\cdot z(\hat{\rho}_{XY\cdot\mathbf{Z}}) > \Phi^{-1}(1-\alpha/2),$$

where Φ(·) is the cumulative distribution function of a Gaussian distribution with zero mean and unit standard deviation, and N is the sample size. Note that this z-transform is approximate and that the actual distribution of the sample (partial) correlation coefficient is not straightforward.

The distribution is description by Fisher.