# 29 Multinomial Distribution

Let $$E_1,E_2,\ldots,E_k$$ be mutually exclusive and exhaustive events and define a multinomial experiment to have the following characteristics:

1. The experiment consists of $$N$$ indepedendent trials.
2. The outcome of each trial belongs to exactly one $$E_j,\ j=1,2,\ldots,k$$.
3. The probability that an outcome belongs to event $$E_j$$ is $$p_j$$.

Let $$X_{ij}=\left\{ \begin{array}{ll} 1& \rm if\ the\ outcome\ of\ the\ \it i^{th}\ \rm trial\ belongs\ to\ \it E_j.\\ 0& otherwise \end{array} \right.$$

and let $$n_j=\sum\limits_{i=1}^{N}X_{ij}$$. Under these conditions, $$N=\sum\limits_{j=1}^{k}n_j$$.

By Lemma 10.0.1 the number of ways to partition $$N$$ into the $$k$$ events, without respect to order, is $$\frac{N!}{n_1!n_2!\cdots n_k!}$$. So the probabilitiy of any particular outcome of the experiment is

$p(n_1,n_2,\ldots,n_{k-1}) = \frac{N!}{n_1!n_2!\cdots n_{k-1}!n^\prime!} p_1^{n_1}p_2^{n_2}\cdots p_{(k-1)}p^{\prime n^\prime}$

where $$n^\prime = N - n_1 - n_2 - \cdots - n_{k-1}$$ and

$$p^\prime = 1 - p_1 - p_2 - \cdots - p_{k-1}$$.

In other words, the entire distribution is defined by the first $$k-1$$ terms.

## 29.1 Cumulative Distribution Function

$P(n_1,n_2,\ldots,n_{k-1}) = \sum\limits_{n_1=0}^{N} \sum\limits_{n_2=0}^{N-n_1} \cdots \sum\limits_{n_{k-1}=0}^{N^\prime} \frac{N!}{n_1!n_2!\cdots n_{k-1}!n^\prime!} p_1^{n_1}p_2^{n_2}\cdots p_{k-1}^{n_{k-1}}p^{\prime n^\prime}$

where $$N^\prime=N-n_1-n_2-\cdots n_{k-1}$$.

## 29.2 Expected Values

Since this is a multivariate distribution, we discuss finding the expected values for each variate $$n_j$$ as opposed to an overall mean.\

$$n_j$$ is a random variable from a multinomial distribution that specifies how many of the $$N$$ observations were of type $$j$$. Each of the $$N$$ observations willf all into exactly one type, so we can conclude that an observation is either of type $$j$$ or it isn’t. Also, it is of type $$j$$ with probability $$p_j$$, and each trial is independent. Thus, we may consider $$n_j$$ a binomial random variable and $$E(n_j)=Np_j$$ and $$V(n_j)=Np_j(1-p_j)$$. Now we must derive the Covariance of $$n_j$$.

We begin by defnining the random variables for $$j\neq m$$:

$X_i=\left\{ \begin{array}{ll} 1 & \rm if\ trial \it\ i\ \rm results\ in\ type\ \it j.\\ 0 & otherwise \end{array} \right.$

$Y_i=\left\{ \begin{array}{ll} 1 & \rm if\ trial \it\ i\ \rm results\ in\ type\ \it m.\\ 0 & otherwise \end{array} \right.$

and let $$n_j=\sum\limits_{i=1}^{n}X_i$$ and $$n_m=\sum\limits_{i=1}^{n}Y_i$$. Since $$X_i$$ and $$Y_i$$ cannot simultaneously equal 1, $$X_i\cdot Y_i=0$$ for all $$i$$. We thus have the following results so far:

$E(X_i\cdot Y_i) = 0$

$E(X_i) = p_j$

$E(Y_i) = p_m$

$$Cov(X_i,Y_i) = 0$$ if $$i\neq j$$ because the trials are independent

$$Cov(X_i,Y_i) = E(X_i\cdot Y_i) - E(X_i)E(Y_i) = 0 - p_j p_m = -p_j p_m$$. ($$Cov(X,Y) = E(XY) - E(X)E(Y)$$ (Theorem 12.2.2)

Using these results we find the Covariance of $$n_j$$ and $$n_m$$.

\begin{aligned} Cov(n_j,n_m) &= \sum\limits_{j=1}^{N}\sum\limits_{m=1}^{N}Cov(X_i,Y_i) \\ &= \sum\limits_{i=1}^{N}Cov(X_iY_i) + \sum\sum\limits_{i\neq j}Cov(X_i,Y_i) \\ &= \sum\limits_{i=1}^{n}-p_jp_m + \sum\sum\limits_{i\neq j}0=-np_jp_m \end{aligned}

The Expected Values of the $$p_j$$’s can be found by

\begin{aligned} E(\hat p_j) &= E(\frac{n_j}{N}) \\ &= \frac{1}{N}E(n_j) \\ &= \frac{1}{N}Np_j \\ &= p_j \\ \\ \\ V(\hat p_j) &= V(\frac{n_j}{N}) \\ &= \frac{1}{N^2}V(n_j) \\ &= \frac{1}{N^2}Np_j(1-p_j) \\ &= \frac{p_j(1-p_j)}{N} \\ \\ \\ Cov(\hat p_j,\hat p_m) &= Cov(\frac{n_j}{N},\frac{n_m}{N}) \\ &= \frac{1}{N^2}Cov(n_j,n_m) \\ &= \frac{1}{N^2}(-Np_jp_m) \\ &= \frac{-p_jp_m}{N} \end{aligned}