# 9 Chi-Square Distribution

## 9.1 Probability Distribution Function

A random variable $$X$$ is said to have a Chi-Square Distribution with parameter $$\nu$$ if its probability distribution function is

$f(x) = \left\{ \begin{array}{ll} \frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} & 0<x,\ 0<\nu\\ 0 & otherwise \end{array} \right.$

$$\nu$$ is commonly referred to as the degrees of freedom.

## 9.2 Cumulative Distribution Function

The cumulative distribution function for the Chi-Square Distribution cannot be written in closed form. It’s integral form is expressed as $F(x) = \left\{ \begin{array}{ll} \displaystyle\int\limits_{0}^{x} \frac{t^{\frac{\nu}{2}-1}e^{-\frac{t}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} dt & 0<x,\ 0<\nu\\\\ 0 & otherwise \end{array} \right.$

## 9.3 Expected Values

\begin{aligned} E(X) &= \int\limits_{0}^{\infty}x\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x\cdot x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}}e^{-\frac{x}{2}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[\Gamma\Big(\frac{\nu}{2}+1\Big)2^{\frac{\nu}{2}+1}\Big] \\ &= \frac{\Gamma(\frac{\nu}{2}+1)2^{\frac{\nu}{2}+1}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{\frac{\nu}{2}\Gamma(\frac{\nu}{2})2^{\frac{\nu}{2}+1}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{2\nu}{2} \\ &= \nu \end{aligned}

1. $$\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)$$

\begin{aligned} E(X^2) &= \int\limits_{0}^{\infty}x^2\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^2\cdot x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}+1}e^{-\frac{x}{2}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[\Gamma(\frac{\nu}{2}+2)2^{\frac{\nu}{2}+2}\Big] \\ &= \frac{\Gamma\Big(\frac{\nu}{2}+2\Big)2^{\frac{\nu}{2}+2}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{(\frac{\nu}{2}+1)\Gamma(\frac{\nu}{2}+1)2^{\frac{\nu}{2}+2}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{\Big(\frac{\nu}{2}+1\Big)\frac{\nu}{2}\Gamma(\frac{\nu}{2})2^{\frac{\nu}{2}+2}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \Big(\frac{\nu}{2}+1\Big)\frac{\nu}{2}\cdot 2^2=2\Big(\frac{\nu}{2}+1\Big)\nu \\ &= (\nu+2)\nu=\nu^2+2\nu \end{aligned}

1. $$\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)$$

\begin{aligned} \mu &= E(X) \\ &= \nu \\ \\ \\ \sigma^2 &= E(X^2)-E(X)^2 \\ &= \nu^2+2\nu-\nu^2 \\ &= 2\nu \end{aligned}

## 9.4 Moment Generating Function

\begin{aligned} M_X(t) &= E(e^{tX}) \\ &= \int\limits_{0}^{\infty}e^{tx} \frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}e^{tx}\cdot x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{tx}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{tx-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{\frac{2tx}{2}-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{-\frac{2tx-x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{-x\frac{-2t+1}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{-x\frac{1-2t}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{\frac{-x}{\frac{2}{1-2t}}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[\Big(\frac{2}{1-2t}\Big)^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})\Big]\\ &= \frac{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})(1-2t)^{\frac{\nu}{2}}} \\ &= \frac{1}{(1-2t)^{\frac{\nu}{2}}} \\ &= (1-2t)^{-\frac{\nu}{2}} \end{aligned}

1. $$\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)$$

\begin{aligned} M_X^{(1)}(t) &= -\frac{\nu}{2}(1-2t)^{-\frac{\nu}{2}-1}(-2) \\ &= \frac{2\nu}{2}(1-2t)^{-\frac{\nu}{2}-1} \\ &= \nu(1-2t)^{-\frac{\nu}{2}-1} \\ \\ \\ M_X^{(2)}(t) &= (-\frac{\nu}{2}-1)\nu(1-2t)^{-\frac{\nu}{2}-2}(-2) \\ &= (\frac{2\nu}{2}+2)\nu(1-2t)^{-\frac{\nu}{2}-2} \\ &= (\nu+2)\nu)(1-2t)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu)(1-2t)^{-\frac{\nu}{2}-2}\\ \\ \\ M_X^{(1)}(0) &= \nu(1-2\cdot 0)^{-\frac{\nu}{2}-1} \\ &= \nu(1-0)^{-\frac{\nu}{2}-1} \\ &= \nu(1)^{-\frac{\nu}{2}-1} \\ &= \nu \\ M_X^{(2)}(0) &= (\nu^2+2\nu)(1-2\cdot 0)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu)(1-0)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu)(1)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu) \\ \\ \\ E(X) &= M_X^{(1)}(0) \\ &= \nu\\ \\ \\ E(X^2) &= M_X^{(2)}(0) \\ &= (\nu^2+2\nu) \\ \\ \\ \mu &= E(X) \\ &= \nu \\ \sigma^2 &= E(X^2)-E(X)^2 \\ &= \nu^2+2\nu-\nu^2 \\ &= 2\nu \end{aligned}

## 9.5 Maximum Likelihood Function

Let $$x_1,x_2,\ldots,x_n$$ be a random sample from a Chi-square distribution with parameter $$\nu$$.

### 9.5.1 Likelihood Function

\begin{aligned} L(\theta) &= f(x_1|\theta) f(x_2|\theta) \cdots f(x_n|\theta) \\ &= \frac{x_1^{\nu/2-1}e^{-x_1/2}}{2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \cdot \frac{x_2^{\nu/2-1}e^{-x_2/2}}{2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \cdots \frac{x_n^{\nu/2-1}e^{-x_n/2}}{2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \\ &= \prod\limits_{i=1}^{n}\frac{x_i^{\nu/2-1}e^{-x_i/2}} {2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \\ &= \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \prod\limits_{i=1}^{n}x_i^{\nu/2-1}e^{-x_i/2} \\ &= \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \cdot \exp\bigg\{ \sum\limits_{i=1}^{n}\frac{x_i}{2} \bigg\} \cdot \prod\limits_{i=1}^{n}x_i^{\nu/2-1} \\ &= \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \cdot \exp\bigg\{ \frac{1}{2}\sum\limits_{i=1}^{n}x_i \bigg\} \cdot \prod\limits_{i=1}^{n}x_i^{\nu/2-1} \end{aligned}

### 9.5.2 Log-likelihood Function

\begin{aligned} \ell(\theta) &= \ln\big(L(\theta)\big) \\ &= \ln\Bigg[ \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \cdot \exp\bigg\{ \frac{1}{2}\sum\limits_{i=1}^{n}x_i \bigg\} \cdot \prod\limits_{i=1}^{n}x_i^{\nu/2-1} \Bigg] \\ &= \ln\Bigg[ \bigg( 2^{\nu/2}\Gamma \Big( \frac{\nu}{2} \Big) \bigg) \Bigg] + \ln\Bigg( \exp\bigg\{ \frac{1}{2}\sum\limits_{i=1}^{n}x_i \bigg\} \Bigg) + \ln\bigg(\prod\limits_{i=1}^{n}x_i^{\nu/2-1}\bigg) \\ &= -n \ln\bigg( 2^{\nu/2}\Gamma \Big( \frac{\nu}{2} \Big) \bigg) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \ln\bigg( \prod\limits_{i=1}^{n}x_i \bigg) \\ &= -n\bigg( \ln(2^{\nu/2}) + \Gamma\Big(\frac{\nu}{2}\Big) \bigg) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \sum\limits_{i=1}^{n}\ln x_i \\ &= -n\bigg(\frac{\nu}{2} \ln 2 + \ln \Gamma\Big( \frac{\nu}{2} \Big) \bigg) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \sum\limits_{i=1}^{n}\ln x_i \\ &= -\frac{n\nu}{2} \ln 2 - n\ln \Gamma\Big( \frac{\nu}{2} \Big) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \sum\limits_{i=1}^{n}\ln x_i \end{aligned}

### 9.5.3 MLE for $$\nu$$

\begin{aligned} \frac{d\ell}{d\nu} &= -\frac{n}{2} \ln 2 - \frac{n}{\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big) \cdot \frac{1}{2} + 0 + \frac{1}{2} \sum\limits_{i=1}^{n}\ln x_i \\ &= -\frac{n}{2} \ln 2 - \frac{n}{2\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big) + \frac{1}{2} \sum\limits_{i=1}^{n}\ln x_i \\ \\ \\ 0 &= -\frac{n}{2} \ln 2 - \frac{n}{2\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big) + \frac{1}{2} \sum\limits_{i=1}^{n}\ln x_i\\ \Rightarrow \frac{n}{2} \ln 2 - \frac{1}{2}\sum\limits_{i=1}^{n}\ln x_i &= -\frac{n}{2\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big)\\ \Rightarrow n\ln 2 - \sum\limits_{i=1}^{n}\ln x_i &= -\frac{n}{\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big)\\ \Rightarrow \frac{\sum\limits_{i=1}^{n}\ln x_i - n\ln 2}{n} &= \frac{\Gamma^\prime\big(\frac{\nu}{2}\big)}{\Gamma\big(\frac{\nu}{2}\big)} \end{aligned}

Due to the complexity of the Gamma function in this equation, no solution can be developed for $$\nu$$ in closed form. Thus, we have to rely on numerical methods to obtain a solution to the equation and find the maximum likelihood estimator.

## 9.6 Theorems for the Chi-Square Distribution

### 9.6.1 Validity of the Distribution

$\int\limits_{0}^{\infty}\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx = 1$

Proof:

\begin{aligned} \int\limits_{0}^{\infty}\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})\Big] \\ &= \frac{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}{2^{\frac{\nu}{2}} \Gamma(\frac{\nu}{2})} \\ &= 1 \end{aligned}

1. $$\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)$$

### 9.6.2 Sum of Chi-Square Random Variables

Let $$X_1 , X_2 , \ldots , X_n$$ be independent Chi-Square random variables with parameter $$\nu_i$$, that is $$X_i\sim\chi^2(\nu_i),\ i=1,2,\ldots,n$$.

Suppose $$Y = \sum\limits_{i=1}^{n}X_i$$. Then $$Y\sim\chi^2(\sum\limits_{i=1}^{n}\nu_i)$$.

Proof:

\begin{aligned} M_Y(t) &= E(e^{tY}=E(e^{t(X_1+X_2+\cdots+X_n}) \\ &= E(e^{tX_1}e^{tX_2}\cdots e^{tX_n}) \\ &= E(e^{tX_1})E(e^{tX_2})\cdots E(e^{tX_n}) \\ &= (1-2t)^{-\frac{\nu_1}{2}}(1-2t)^{-\frac{\nu_2}{2}}\cdots (1-2t)^{-\frac{\nu_n}{2}} \\ &= (1-2t)^{\sum\limits_{i=1}^{n}\nu_i} \end{aligned}

Which is the mgf of a Chi-Square random variable with parameter $$\sum\limits_{i=1}^{n}\nu_i$$.
Thus $$Y\sim\chi^2\bigg(\sum\limits_{i=1}^{n}\nu_i\bigg)$$.

### 9.6.3 Multiple of a Chi-Square Random Variable

Let $$X$$ be a Chi-Square random variable with parameter $$\nu$$, that is $$X\sim\chi^2(\nu),\ i=1,2,\ldots,n$$.

Suppose $$Y = c \cdot X$$. Then $$Y\sim Gamma(\frac{\nu}{2}, 2 \cdot c)$$.

Proof:

\begin{aligned} M_Y(t) &= E(e^{tY})=E(e^{tcX}) \\ &= (1-2tc)^{-\frac{\nu}{2}} \\ &= (1-2c \cdot t)^{-\frac{\nu}{2}} \end{aligned}

Which is the mgf of a Gamma distributed variable with parameters $$\alpha = \frac{\nu}{2}$$ and $$\beta = 2c$$. Thus, $$Y\sim Gamma(\frac{\nu}{2}, 2 \cdot c)$$.

### 9.6.4 Square of a Standard Normal Random Variable

If $$Z\sim N(0,1)$$, then $$Z^2\sim\chi^2(1)$$.

Proof: \begin{aligned} M_{Z^2}(t) &= E(e^{tZ^2}) \\ &= \int\limits_{-\infty}^{\infty}e^{tz^2}\frac{1}{\sqrt{2\pi}} e^{-\frac{z^2}{2}}dz \\ &= \frac{1}{\sqrt{2\pi}}\int\limits_{-\infty}^{\infty}e^{tz^2} e^{-\frac{z^2}{2}}dz \\ &= \frac{1}{\sqrt{2\pi}}\int\limits_{-\infty}^{\infty} e^{-\frac{z^2}{2}(-2t+1)}dz \\ &= \frac{1}{\sqrt{2\pi}}\int\limits_{-\infty}^{\infty} e^{-\frac{z^2}{2}(1-2t)}dz \\ ^{[1]} &= \frac{2}{\sqrt{2\pi}}\int\limits_{0}^{\infty} e^{-\frac{z^2}{2}(1-2t)}dz \\ ^{[2]} &= \frac{2}{\sqrt{2\pi}}\int\limits_{0}^{\infty}e^{-u} \frac{\sqrt{2}u^{-\frac{1}{2}}}{2(1-2t)^{\frac{1}{2}}}du \\ &= \frac{2\sqrt{2}}{2\sqrt{2\pi}(1-2t)^{\frac{1}{2}}} \int\limits_{0}^{\infty}e^{-u}u^{-\frac{1}{2}}du \\ &= \frac{2\sqrt{2}}{2\sqrt{2\pi}(1-2t)^{\frac{1}{2}}} \int\limits_{0}^{\infty}u^{\frac{1}{2}-1}e^{-u}du \\ ^{[3]} &= \frac{1}{\sqrt{\pi}(1-2t)^{\frac{1}{2}}}\Gamma(\frac{1}{2}) \\ &= \frac{\sqrt{\pi}}{\sqrt{\pi}(1-2t)^{\frac{1}{2}}} \\ &= \frac{1}{(1-2t)^{\frac{1}{2}}}=(1-2t)^{-\frac{1}{2}} \\ \end{aligned}

1. $$\int\limits_{-\infty}^{\infty}f(x)dx = 2\int\limits_{0}^{\infty}f(x)dx$$ when f(x) is an even function ()
2. Let $$u=\frac{z^2}{2}(1-2t) \ \ \ \ \Rightarrow z=\frac{\sqrt{2}u^{\frac{1}{2}}}{(1-2t)^{\frac{1}{2}}}$$
So $$dz=\frac{\sqrt{2}u^{-\frac{1}{2}}} {2(1-2t)^{\frac{1}{2}}}$$
3. $$\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx =\beta^\alpha\Gamma(\alpha)$$

Which is the mgf of a Chi-Square random variable with 1 degree of freedom. Thus $$Z^2\sim\chi^2(1)$$.