# What is Covariance?

Probability / Thursday, September 27th, 2018

# Covariance Definition

Covariance is a measure of association between two random variables. Let X and Y be two random variables. Then the covariance is defined as
$Cov\left( X,Y \right)=E\left[ \left\{ X-E\left( X \right) \right\}\left\{ Y-E\left( Y \right) \right\} \right]$
$\Rightarrow Cov\left( X,Y \right)=E\left[ XY-XE\left( Y \right)-YE\left( X \right)+E\left( X \right)E\left( Y \right) \right]$
$\Rightarrow Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)-E\left( X \right)E\left( Y \right)+E\left( X \right)E\left( Y \right)$
$\therefore Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)$

Note:

If X and Y are independent random variables, then Cov(X, Y) = 0

## Properties of Covariance:

$(i)~Cov\left( aX,bY \right)=abCov\left( X,Y \right)$
$(ii)Cov\left( X+a,Y+b \right)=Cov\left( X,Y \right)$
$(iii)Cov\left( \frac{X-\overline{X}}{\sigma X},\frac{Y-\overline{Y}}{\sigma Y} \right)=\frac{1}{\sigma X\sigma Y}$
$(iv)Cov\left( aX+b,cY+d \right)=acCov\left( X,Y \right)$
$(v)Cov\left( X+Y,Z \right)=Cov\left( X,Z \right)+Cov\left( Y,Z \right)$
$(vi)Cov\left( aX+bY,cX+dY \right)=ac{{\sigma }_{X}}^{2}+bd{{\sigma }_{Y}}^{2}+\left( ad+bc \right)Cov\left( X,Y \right)$
$(vii)\text{If}~~{{X}_{1}},{{X}_{2}},…,{{X}_{n}}~~\text{be n random variables},\text{ then}$
$\text{V}\left( \sum\nolimits_{i=1}^{n}{{{a}_{i}}{{X}_{i}}} \right)=\sum\nolimits_{i=1}^{n}{{{a}_{i}}^{2}V\left( {{X}_{i}} \right)}+2\sum\nolimits_{i=1}^{n}{\sum\nolimits_{j=1}^{n}{{{a}_{i}}{{a}_{j}}}}Cov\left( {{X}_{i}},{{X}_{j}} \right)$
$(viii)\text{If}~~{{X}_{1}},{{X}_{2}},…,{{X}_{n}}~~\text{be n independent random variables},\text{ then}$
$\text{V}\left( \sum\nolimits_{i=1}^{n}{{{a}_{i}}{{X}_{i}}} \right)=\sum\nolimits_{i=1}^{n}{{{a}_{i}}^{2}V\left( {{X}_{i}} \right)}$

 Example 01

The score of 12 students in mathematics and physics classes are

 M 2 3 4 4 5 6 6 7 7 8 10 10 P 1 3 2 4 4 4 6 4 6 7 9 10

Find the covariance of the distribution.

Solution:

We know that,
$Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)$
Now do it yourself. You can take the help of “mathematical expectation” article to solve this sum.

 Example 02

Let X and Y be two random variables such that Var(X) = 4, Cov(X, Y) = 2. Compute the Cov(3X, X+3Y).

Solution:

By the properties of covariance operator we have,
$Cov\left( 3X,X+3Y \right)=3Cov\left( X,X+3Y \right)$
$\Rightarrow Cov\left( 3X,X+3Y \right)=3Cov\left( X,X \right)+9Cov\left( X,Y \right)$
$\Rightarrow Cov\left( 3X,X+3Y \right)=3Var\left( X \right)+9Cov\left( X,Y \right)$
$\therefore Cov\left( 3X,X+3Y \right)=3\times 4+9\times 2=30$

 Example 03

Let (X, Y) be an absolutely continuous random variable with support RXY = [0, ∞) X [1, 4] and its joint probability density function be
${{f}_{XY}}=\frac{1}{3}y.\exp \left( -xy \right),if~~x\in [0,\infty )\And y\in [1,4]$
${{f}_{XY}}=0,~~otherwise$
Compute covariance between X and Y.

Solution:

Note that the support of Y is RY = [1, 4]. When y does not belongs to [1, 4], the marginal probability density function of Y is 0, while yЄ[1, 4], the marginal probability density function of Y is
${{f}_{Y}}\left( y \right)=\int\limits_{-\infty }^{\infty }{{{f}_{XY}}\left( x,y \right)}dx$
$\Rightarrow {{f}_{Y}}\left( y \right)=\int\limits_{0}^{\infty }{\frac{1}{3}y.\exp \left( -xy \right)}dx$
$\Rightarrow {{f}_{Y}}\left( y \right)=\frac{1}{3}y.\left[ \frac{\exp \left( -xy \right)}{-y} \right]_{0}^{\infty }$
$\therefore {{f}_{Y}}\left( y \right)=\frac{1}{3}\left[ 0-\left( -1 \right) \right]=\frac{1}{3}$
The expected value of Y is
$E\left( Y \right)=\int\limits_{-\infty }^{\infty }{y}{{f}_{Y}}\left( y \right)dy=\int\limits_{1}^{4}{y.}\frac{1}{3}dy=\frac{1}{3}\left[ \frac{{{y}^{2}}}{2} \right]_{1}^{4}=\frac{15}{6}$
Now, the support of X is RX = [0, ∞). When y does not belongs to [0, ∞), the marginal probability density function of X is 0, while xЄ[0, ∞), the marginal probability density function of X is
${{f}_{x}}\left( x \right)=\int\limits_{-\infty }^{\infty }{{{f}_{XY}}\left( x,y \right)dy}$
$\Rightarrow {{f}_{x}}\left( x \right)=\int\limits_{1}^{4}{\frac{1}{3}y.\exp \left( -xy \right)dy}$
The expected value of X is
$E\left( X \right)=\int\limits_{-\infty }^{\infty }{x{{f}_{x}}\left( x \right)}dx=\int\limits_{0}^{\infty }{x\left\{ \int\limits_{1}^{4}{\frac{1}{3}y.\exp \left( -xy \right)dy} \right\}}dx$
$\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ \int\limits_{0}^{\infty }{xy.\exp \left( -xy \right)dx} \right\}}dy$
$\left( \text{by changing the order of integration} \right)$
$\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ \frac{1}{y}\int\limits_{0}^{\infty }{t\exp (-t)dt} \right\}}dy$
$\left( \text{by changing the variable in the inner integral},\text{ t }=\text{ xy} \right)$
$\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\frac{1}{y}\left\{ \left[ -t\exp (-t) \right]_{0}^{\infty }+\int\limits_{0}^{\infty }{\exp (-t)dt} \right\}}dy$
$\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\frac{1}{y}\left\{ 0+\left[ \exp (-t) \right]_{0}^{\infty } \right\}}dy$
$\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\frac{1}{y}dy=}\frac{1}{3}\left[ \log y \right]_{1}^{4}=\frac{1}{3}\log 4$
The expected value of the product XY can be computed using the transformation theorem
$E\left( XY \right)=\int\limits_{-\infty }^{\infty }{\int\limits_{-\infty }^{\infty }{xy~~{{f}_{xy}}\left( xy \right)dydx}}$
$\Rightarrow E\left( XY \right)=\int\limits_{0}^{\infty }{\left\{ \int\limits_{1}^{4}{xy.}\frac{1}{3}y.\exp \left( -xy \right)dy \right\}dx}$
$\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{y\left\{ \int\limits_{0}^{\infty }{xy.}\exp \left( -xy \right)dx \right\}}dy$
$\left( \text{by changing the order of integration} \right)$
$\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{y\left\{ \frac{1}{y}\int\limits_{0}^{\infty }{t.}\exp \left( -t \right)dt \right\}}dy$
$\left( \text{by changing the variable in the inner integral},\text{ t }=\text{ xy} \right)$
$\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ \left[ t\exp \left( -t \right) \right]_{0}^{\infty }+\int\limits_{0}^{\infty }{\exp \left( -t \right)d}t \right\}}dy$
$\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ 0+\left[ -\exp \left( -t \right) \right]_{0}^{\infty } \right\}}dy$
$\therefore E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{dy=}\frac{1}{3}\left[ y \right]_{1}^{4}=1$
Hence, using the covariance formula, the covariance between X and Y is
$Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)$
$\therefore Cov\left( X,Y \right)=1-\frac{1}{3}\log 4.\frac{15}{6}=1-\frac{5}{6}\log 4$

 Example 04

Let [X, Y] be an absolutely continuous random vector with support RXY = {(x, y): 0≤x≤y≤2} i.e., RXY is the set of all couples (x, y) such that 0≤y≤2 & 0≤x≤y. Let the joint probability density function of [X, Y] be
${{f}_{XY}}\left( x,y \right)=\frac{3}{8}y,~~if~~(x,y)\in {{R}_{XY}}$
${{f}_{XY}}=0,~~otherwise$
Compute the covariance between X and Y.

Solution:

Do it yourself, like above example.

<< Previous  Next>>