Multinomial Distribution

6.3. Multinomial Distribution#

Bernoulli trials come out in one of two ways. But many trials come out in multiple different ways, all of which we might want to track. A die can land six different ways. A jury member can have one of several different educational levels. In general, an individual might belong to one of several classes.

The multinomial distribution is a joint distribution that extends the binomial to the case where each repeated trial has more than two possible outcomes. Let’s look at it first in an example, and then we will define it in general.

A box contains 2 blue tickets, 5 green tickets, and 3 red tickets. Fifteen draws are made at random with replacement. To find the chance that there are 4 blue, 9 green, and 2 red tickets drawn, we could start by writing all possible sequences of 4 B’s, 9 G’s, and 2 R’s.

Each such sequence has chance ${0.2}^{4} {0.5}^{9} {0.3}^{2}$ , so all we need for completing the probability calculation is the number of sequences we could write.

There are $(\binom{15}{4})$ ways of choosing places to write the B’s.
For each of these ways, there are $(\binom{11}{9})$ ways of choosing 9 of the remaining 11 places to write the G’s.
The remaining 2 places get filled with R’s.

So

\begin{array}{r} \begin{aligned} P (4 blue, 9 green, 2 red) & = (\binom{15}{4}) \cdot (\binom{11}{9}) {0.2}^{4} {0.5}^{9} {0.3}^{2} \\ = \frac{15!}{4! 11!} \cdot \frac{11!}{9! 2!} {0.2}^{4} {0.5}^{9} {0.3}^{2} \\ = \frac{15!}{4! 9! 2!} {0.2}^{4} {0.5}^{9} {0.3}^{2} \end{aligned} \end{array}

Notice how this simply extends the binomial probability formula by including a third category.

Analogously, or formally by induction, you can extend the formula to any finite number of categories or classes.

6.3.1. Multinomial Distribution#

Fix a positive integer $n$ . Suppose we are running $n$ i.i.d. trials where each trial can result in one of $k$ classes. For each $i = 1, 2, \dots, k$ , let the chance of getting Class $i$ on a single trial be $p_{i}$ , so that $\sum_{i = 1}^{k} p_{i} = 1$ .

For each $i = 1, 2, \dots, k$ , let $N_{i}$ be the number of trials that result in Class $i$ , so that $\sum_{i = 1}^{k} N_{i} = n$ .

Then the joint distribution of $N_{1}, N_{2}, \dots, N_{k}$ is given by

P (N_{1} = n_{1}, N_{2} = n_{2}, \dots, N_{k} = n_{k}) = \frac{n!}{n_{1}! n_{2}! \dots n_{k}!} p_{1}^{n_{1}} p_{2}^{n_{2}} \dots p_{k}^{n_{k}}

where $n_{i} \geq 0$ for $1 \leq i \leq k$ and $\sum_{i = 1}^{k} n_{i} = n$ .

This is called the multinomial distribution with parameters $n$ and $p_{1}, p_{2}, \dots, p_{k}$ .

When there are just two classes, then $k = 2$ and the formula reduces to the familiar binomial formula written as the joint distribution of the number of successes and the number of failures:

P (N_{1} = n_{1}, N_{2} = n_{2}) = \frac{n!}{n_{1}! n_{2}!} p_{1}^{n_{1}} p_{2}^{n_{2}} where p_{1} + p_{2} = 1 and n_{1} + n_{2} = n

Quick Check

In a population, 20% of the individuals are in Class A, 50% in Class B, and 30% in Class C. If 20 individuals are drawn at random with replacement, what is the chance that the sample contains 3 individuals in Class A, 13 in Class B, and 4 in Class C?

Answer

$\frac{20!}{3! 13! 4!} {0.2}^{3} {0.5}^{13} {0.3}^{4}$

6.3.2. Binomial Marginals#

No matter how many classes there are, the marginal distribution of each $N_{i}$ is binomial $(n, p_{i})$ .

You don’t have to sum terms in the joint distribution to work this out.

$N_{i}$ is the number of Class $i$ individuals in the sample
Each sampled individual is in Class $i$ with probability $p_{i}$
There are $n$ independent draws.

That’s the binomial setting.

6.3.3. A Roulette Example#

A Nevada roulette wheel has 18 red pockets, 18 black pockets, and 2 green pockets. Each time the wheel is spun, all 38 pockets are equally likely to win, independent of all other spins.

Question: If the wheel is spun 10 times, what is the chance that red and black win an equal number of times?

Answer: Let $N_{r}$ be the number of times red wins, $N_{b}$ the number of times black wins, and $N_{g}$ the number of times green wins.

\begin{array}{r} \begin{aligned} P (N_{r} = N_{b}) & = \sum_{i = 0}^{5} P (N_{r} = i, N_{b} = i, N_{g} = 10 - 2 i) \\ = \sum_{i = 0}^{5} \frac{10!}{i! i! (10 - 2 i)!} (\frac{18}{38})^{i} (\frac{18}{38})^{i} (\frac{2}{38})^{10 - 2 i} \end{aligned} \end{array}

Multinomial Distribution

Contents

6.3. Multinomial Distribution#

6.3.1. Multinomial Distribution#

6.3.2. Binomial Marginals#

6.3.3. A Roulette Example#