# 12 Covariance

## 12.1 Definition of Covariance

For any two random variables $$X$$ and $$Y$$, the covariance of $$X$$ and $$Y$$ is defined as

$\mathrm{Cov}(X,Y) = E[(X-\mu_X)(Y-\mu_Y)]$

## 12.2 Theorems on Covariance

### 12.2.1 Theorem

Let $$X$$ be a random variable. Then $\mathrm{Cov}(X,X) = V(X)$

Proof:

\begin{aligned} \mathrm{Cov}(X,X) &= E[(X-\mu)(X-\mu)] \\ &= E[(X-\mu)^2] \\ &= V(X) \end{aligned}

### 12.2.2 Theorem

Let $$X$$ and $$Y$$ be random variables. Then

$\mathrm{Cov(X,Y)} = E(XY)-E(X)E(Y)$

Proof:

\begin{aligned} \mathrm{Cov(X,Y)} &= E[(X-\mu_x)(Y-\mu_Y)] \\ &= E[XY - X\mu_y - Y\mu_X + \mu_X\mu_Y] \\ &= E(XY) - E(X)\mu_Y - \mu_XE(Y) + \mu_X\mu_Y \\ &= E(XY) - E(X)E(Y) - E(X)E(Y) + E(X)E(Y) \\ &= E(XY) - 2E(X)E(Y) + E(X)E(Y) \\ &= E(XY) - E(X)E(Y) \end{aligned}

### 12.2.3 Covariance

Let $$X$$ and $$Y$$ be random variables and let $$a$$ and $$b$$ be constants. Then

$\mathrm{Cov(aX,bY)} = ab \mathrm{Cov}(X,Y)$

Proof:

\begin{aligned} \mathrm{Cov}(aX,bY) &= E(aXbY) - E(aX)E(bY) \\ &= abE(XY) - abE(X)E(Y) \\ &= ab[E(XY) - E(X)E(Y)] \\ &= ab \cdot \mathrm{Cov}(X,Y) \end{aligned}

### 12.2.4 Theorem

Let $$X_1 , X_2 , \ldots , X_n$$ be random variables with $$E(X_i) = \mu_i$$ for $$i = 1,2,\ldots,n$$ and let $$Y_1,Y_2,\ldots,Y_m$$ be random variables with $$E(Y_j) = \phi_j$$ for $$j=1,2,\ldots,m$$. Also, let $$a_1,a_2,\ldots,a_n$$ and $$b_1,b_2,\ldots,b_m$$ be constants.\

If $$U_1 = \sum\limits_{i=1}^{n}a_iX_i$$ and $$U_2 = \sum\limits_{i=1}^{m}b_iY_i$$, then

$\mathrm{Cov}(U_1,U_2) = \sum\limits_{i=1}^{n}\sum\limits_{j=1}^{m}a_ib_j \cdot \mathrm{Cov}(X_i,Y_j)$

Proof:

\begin{aligned} \mathrm{Cov}(U_1,U_2) &= E[(U_1-E(U_1))(U_2-E(U_2))] \\ &= E\Big[\big(\sum\limits_{i=1}^{n}a_iX_i-\sum\limits_{i=1}^{n}a_i\mu_i\big) \big(\sum\limits_{j=1}^{m}b_jY_j-\sum\limits_{j=1}^{m}b_j\phi_j\big)\Big] \\ &= E\Big[\big(\sum\limits_{i=1}^{n}a_i(X_i-\mu_i)\big)\big(\sum\limits_{j=1}^{m}b_j(Y_j-\phi_j)\big)\Big] \\ &= E\Big[\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{m}a_ib_j(X_i-\mu_i)(Y_j-\phi_j)\Big] \\ &= \sum\limits_{i=1}^{n}\sum\limits_{j=1}^{m}a_ib_jE[(X_i-\mu_i)(Y_j-\phi_j)] \\ &= \sum\limits_{i=1}^{n}\sum\limits_{j=1}^{m}a_ib_j\ \mathrm{Cov}(X_i,Y_j) \end{aligned}

### 12.2.5 Theorem

Let $$X_1,X_2,\ldots,X_n$$ be random variables with $$E(X_i)=\mu_i$$ for $$i=1,2,\ldots,n$$ and let $$a_1,a_2,\ldots,a_n$$ be constants.

If $$Y = \sum\limits_{i=1}^{n}a_iX_i$$ then

$V(Y) = \sum\limits_{i=1}^{n}a_i^2V(X_i)+2\sum\limits_{\ \ i<}\sum\limits_{j\ \ }a_ia_j\ \mathrm{Cov}(X_i,X_j)$

Proof:

\begin{aligned} V(Y) &= E[(Y-\mu_Y)^2] \\ &= E[(Y-\mu_Y)(Y-\mu_Y)] \\ &= E\Big[\big(\sum\limits_{i=1}^{n}a_iX_i-a_i\mu_i\big) \big(\sum\limits_{n=1}^{n}a_jX_j-a_j\mu_j\big)\Big] \\ &= \sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}a_ia_jE[(X_i-\mu_i)(X_j-\mu_j)] \\ &= \sum\limits_{i=1}^{n}a_i^2\ \mathrm{Cov}(X_i,X_i)+ \sum\limits_{\ \ i\neq}\sum\limits_{j\ \ \ \ }a_ia_jE[(X_i-\mu_i)(X_j-\mu_j)] \\ &= \sum\limits_{i=1}^{n}a_i^2V(X_i)+ 2\sum\limits_{\ \ i<}\sum\limits_{j\ \ \ \ }a_ia_j\ \mathrm{Cov}(X_i,X_j) \end{aligned}