# 31 Poisson Distribution

## 31.1 Probability Mass Function

A random variable is said to have a Poisson distribution with parameter $$\lambda$$ if its probability mass function is:

$p(x)=\left\{ \begin{array}{ll} \frac{\lambda^xe^{-\lambda}}{x!}, & x=0,1,2,...\\ 0 & otherwise \end{array}\right.$

## 31.2 Cumulative Mass Function

$P(x)=\left\{ \begin{array}{lll} e^{-\lambda}\sum\limits_{i=0}^{x}\frac{\lambda^i}{i!}, & x=1,2,3,...\\ 0 & otherwise \end{array}\right.$

A recursive form of the cdf can be derived and has some usefulness in computer applications. With it, one need only initiate the first value and additional cumulative probabilities can be calculated. It is derived as follows:

\begin{aligned} P(X=x+1) &= \frac{e^{-\lambda}\lambda^{x+1}}{(x+1)!} \\ &= \frac{\lambda}{x+1}\frac{e^{-\lambda}\lambda^x}{x!} \\ &= \frac{\lambda}{x+1}P(X=x) \end{aligned}

## 31.3 Expected Values

\begin{aligned} E(X) &= \sum\limits_{x=0}^{\infty}x\frac{\lambda^xe^{-\lambda}}{x!} \\ &= e^{-\lambda}\sum\limits_{x=0}^{\infty}x\frac{\lambda^x}{x!} \\ &= e^{-\lambda}\Big(0\frac{\lambda^0}{0!}+1\frac{\lambda^1}{1!} +2\frac{\lambda^2}{2!}+3\frac{\lambda^3}{3!}+\cdots\Big) \\ &= e^{-\lambda}\Big(0+\lambda^1+\frac{\lambda^2}{1!}+\frac{\lambda^3}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\Big(\lambda^0+\frac{\lambda^1}{1!}+\frac{\lambda^2}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\Big(\frac{\lambda^0}{0!}+\frac{\lambda^1}{1!} +\frac{\lambda^2}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}e^\lambda \\ &= \lambda e^(-\lambda+\lambda) \\ &= \lambda \end{aligned}

1. Taylor Series Expansion: $$e^x=\frac{x^0}{0!}+\frac{x^1}{1!} +\frac{x^2}{2!}+\cdots =1+\frac{x^1}{1!}+\frac{x^2}{2!}+\cdots$$

\begin{aligned} E(X^2) &= \sum\limits_{x=0}^{\infty}x\frac{\lambda^xe^{-\lambda}}{x!} \\ &= e^{-\lambda}\sum\limits_{x=0}^{\infty}x\frac{\lambda^x}{x!} \\ &= e^{-\lambda}\Big(0^2\frac{\lambda^0}{0!}+1^2\frac{\lambda^1}{1!} +2^2\frac{\lambda^2}{2!}+3^2\frac{\lambda^3}{3!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\Big(\frac{\lambda^0}{1} +2\frac{\lambda^1}{1!}+3\frac{\lambda^2}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\sum\limits_{x=0}^{\infty}(x+1)\frac{\lambda^x}{x!} \\ &= \lambda e^{-\lambda}\sum\limits_{x=0}^{\infty} \Big(x\frac{\lambda^x}{x!}+\frac{\lambda^x}{x!}\Big) \\ &= \lambda e^{-\lambda}\Big(\sum\limits_{x=0}^{\infty}x\frac{\lambda^x}{x!} +\sum\limits_{x=0}^{\infty}\frac{\lambda^x}{x!}\Big) \\ &= \lambda\sum\limits_{x=0}^{\infty}x\frac{\lambda^xe^{-\lambda}}{x!} +\lambda\sum\limits_{x=0}^{\infty}\frac{\lambda^xe^{-\lambda}}{x!} \\ &= \lambda E(X)+\lambda \\ &= \lambda^2+\lambda\\ \\ \\ \mu &= E(X) \\ &= \lambda \\ \\ \\ \sigma^2 &= E(X^2) - E(X) \\ &= \lambda^2 + \lambda - \lambda^2 \\ &= \lambda \end{aligned}

## 31.4 Moment Generating Function

\begin{aligned} M_X(t) &= E(e^{tX}) \\ &= \sum\limits_{x=0}^{\infty}e^{tx}\frac{\lambda^xe^{-\lambda}}{x!} \\ &= \sum\limits_{x=0}^{\infty}\frac{(\lambda e^{tx})^xe^{-\lambda}}{x!} \\ &= e^{-\lambda}\sum\limits_{x=0}^{\infty}\frac{(\lambda e^{tx})^x}{x!} \\ &= e^{-\lambda}\Big[\frac{(\lambda e^t)^0}{0!}+\frac{(\lambda e^t)^1}{1!}+ \frac{(\lambda e^t)^2}{2!}+\cdots\Big] \\ &= e^{-\lambda}e^{\lambda e^t} \\ &= e^{(\lambda e^t-\lambda)} \\ &= e^{\lambda(e^t-1)} \end{aligned}

1. Taylor Series Expansion: $$e^x=\frac{x^0}{0!}+\frac{x^1}{1!} +\frac{x^2}{2!}+\cdots =1+\frac{x^1}{1!}+\frac{x^2}{2!}+\cdots$$

\begin{aligned} M_X^{(1)}(t) &= e^{\lambda(e^t-1)}\lambda e^t=\lambda e^te^{\lambda(e^t-1)} \\ \\ \\ M_X^{(2)}(t) &= (e^{\lambda(e^t-1)}(\lambda e^t)+(e^{\lambda(e^t-1)}\lambda e^t)(\lambda e^t) \\ &= \lambda e^t[e^{\lambda e^t-1)}+e^{\lambda(e^t-1)}\lambda e^t] \\ &= \lambda e^t[e^{\lambda(e^t-1)}(1+\lambda e^t)] \\ \\ \\ E(X) &= M_X^{(1)}(0) \\ &= \lambda e^0e^{\lambda(e^0-1)}\lambda e^t=\lambda \\ \\ \\ E(X^2) &= M_X^{(2)}(0) \\ &= \lambda e^0[e^{\lambda(e^0-1)}(1+\lambda e^0)] \\ &= \lambda e^0[e^{\lambda(e^0-1)}(1+\lambda e^0] \\ &= \lambda(1+\lambda) \\ &= \lambda+\lambda^2 \\ \\ \\ \mu &= E(X) \\ &= \lambda \\ \sigma^2 &= E(X^2)-E(X) \\ &= \lambda+\lambda^2-\lambda^2 \\ &= \lambda \end{aligned}

## 31.5 Maximum Likelihood Estimator

Let $$x_1,x_2,\ldots,x_n$$ be a random sample drawn from a Poisson Distribution with parameter $$\lambda$$.

### 31.5.1 Likelihood Function

\begin{aligned} L(\theta) &= L(x_1,x_2,\ldots,x_n|\theta) \\ &= p(x_1|\theta)p(x_2|\theta)\cdots p(x_n|\theta) \\ &= \frac{e^{-\lambda}\lambda^{x_1}}{x_1!}\cdot\frac{e^{-\lambda}\lambda^{x_2}}{x_2!} \cdot \ \cdots\ \cdot \frac{e^{-\lambda}\lambda^{x_n}}{x_n!} \\ &= \frac{e^{-n\lambda}\lambda^{\sum\limits_{i=1}^{n}x_i}}{\prod\limits_{i=1}^{n}x_1!} \end{aligned}

### 31.5.2 Log-likelihood

\begin{aligned} \ell(\lambda) &= \ln(L(\lambda)) \\ &= \ln\Big[\frac{e^{-n\lambda}\lambda^{\sum\limits_{i=1}^{n}x_i}}{\prod\limits_{i=1}^{n}x_1!}\Big] \\ &= \ln(e^{-n\lambda})+\ln\Big(\lambda^{\sum\limits_{i=1}^{n}x_i}\Big) - \ln\Big(\prod\limits_{i=1}^{n}x_1!\Big) \\ &= -n\lambda + \sum\limits_{i=1}^{n}x_i\ln\Big(\lambda)-\ln(\prod\limits_{i=1}^{n}x_1!\Big) \end{aligned}

### 31.5.3 MLE for $$\lambda$$

\begin{aligned} \frac{d\ell}{d\lambda} &= -n - \frac{\sum\limits_{i=1}^{n}x_i}{\lambda} - 0 \\ &= \frac{\sum\limits_{i=1}^{n}x_i}{\lambda} - n \\ \\ \\ 0 &= \frac{\sum\limits_{i=1}^{n}x_i}{\lambda}-n\\ \Rightarrow \frac{\sum\limits_{i=1}^{n}x_i}{\lambda} &= n\\ \Rightarrow \sum\limits_{i=1}^{n}x_i &= n\lambda\\ \Rightarrow \frac{\sum\limits_{i=1}^{n}x_i}{n} &= \lambda \end{aligned}

so $$\displaystyle \hat\lambda=\frac{\sum\limits_{i=1}^{n}x_i}{n}$$ is the Maximum Likelihood Estimator for $$\lambda$$.

## 31.6 Theorems for the Poisson Distribution

### 31.6.1 Derivation of the Poisson Distribution

Suppose $$X$$ is a Binomial random variable in all respects but has an infinite (non-fixed) number of trials, each with probability of success $$p$$.

Then the pdf of $$X$$ is $$P(x)=\frac{e^{-\lambda}\lambda^x}{x!}$$

Proof:

For an infinite number of trials we take $$\lim\limits_{n\rightarrow\infty}{n\choose x}p^x(1-p)^{n-x}$$.

By rewriting $$p$$ in terms of $$\mu\ (\mu=np\ \Rightarrow p=\frac{\mu}{n})$$ we get

\begin{aligned} \lim\limits_{n\rightarrow\infty}{n\choose x}\frac{\mu}{n}^x(1-\frac{\mu}{n})^{n-x} &= \lim\limits_{n\rightarrow\infty}{n\choose x} \frac{\mu}{n}^x(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ &= \lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)(n-x)!}{x!(n-x)!}\bigg) \mu^x\frac{1}{n^x}(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ &= \lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)}{x!}\bigg) \mu^x\frac{1}{n^x}(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ &= \frac{\mu^x}{x!}\lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)}{n^x}\bigg) \frac{1}{n^x}(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ ^{[1]}&= \frac{\mu^x}{x!}\lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)}{n^x}\bigg) \lim\limits_{n\rightarrow\infty}\frac{1}{n^x}(1-\frac{\mu}{n})^n \lim\limits_{n\rightarrow\infty}(1-\frac{\mu}{n})^{-x} \\ &= \frac{\mu^x}{x!}\cdot 1\cdot e^{-\mu}\cdot 1 \\ &= \frac{e^{-\mu}\mu^x}{x!} \end{aligned}

1. $$\lim\limits_{n\rightarrow\infty}(1-\frac{x}{n})^n=e^-x$$

Traditionally, we use $$\lambda$$ in place of $$\mu$$ for the Poisson distribution, giving us the desired result. (Wackerly, III, and Scheaffer 2002, 125)

### 31.6.2 Validity of the Distribution

$\sum\limits_{x=0}^{\infty}\frac{e^{-\lambda}\lambda^x}{x!} = 1$

Proof:

\begin{aligned} \sum\limits_{x=0}^{\infty}\frac{e^{-\lambda}\lambda^x}{x!} &= e^{-\lambda}\sum\limits_{x=0}^{\infty}\frac{\lambda^x}{x!} \\ &= e^{-\lambda}\bigg(\frac{\lambda^0}{0!}+ \frac{\lambda^1}{1!}+\frac{\lambda^2}{2!}+\cdots\bigg) \\ &= e^{-\lambda}\cdot e^\lambda \\ &= e^0 \\ &= 1 \end{aligned}

1. Taylor Series Expansion: $$e^x=\frac{x^0}{0!}+\frac{x^1}{1!}+\frac{x^2}{2!}+\cdots$$

### 31.6.3 Sum of Poisson Random Variables

Let $$X_1,X_2,\ldots,X_n$$ be independent random variables from a Poisson distribution with parameter $$\lambda_i,\ i=1,2,\ldots,n$$; that is, $$X_i\sim$$Poisson$$(\lambda_i)$$.

Let $$Y=\sum\limits_{i=1}^{n}X_i$$.

Then $$Y\sim$$Poisson$$(\sum\limits_{i=1}^{n}\lambda_i)$$.\

Proof:

\begin{aligned} M_Y(t) &= E(e^{tY}) \\ &= E(e^{t(X_1+X_2+\cdots+X_n)} \\ &= E(e^{tX_1}e^{tX_2}\cdots e^{tX_n}) \\ &= E(e^{tX_1})E(e^{tX_2})\cdots E(e^{tX_n}) \\ &= e^{\lambda_1(e^t-1)}e^{\lambda_2(e^t-1)}\cdots e^{\lambda_n(e^t-1)} \\ &= e^{(\lambda_1+\lambda_2+\cdots+\lambda_n)(e^t-1)} \\ &= e^{(e^{t-1})\sum\limits_{i=1}^{n}\lambda_i} \end{aligned}

Which is the mgf of a Poisson random variable with parameter $$\sum\limits_{i=1}^{n}\lambda_i$$. Thus $$Y\sim$$Poisson$$(\sum\limits_{i=1}^{n}\lambda_i)$$.

### References

Wackerly, Dennis D., William Mendenhall III, and Richard L. Scheaffer. 2002. Mathematical Statistics with Applications. 6th ed. Duxbury.