Distribution of the Number of Successes


Distribution of the Number of Successes

As in the previous section, let $X$ have the beta $(r, s)$ prior, and given $X = p$ let the $S_n$ be the number of heads in the first $n$ tosses of a $p$-coin.

All the calculations we carried out in the previous section were under the condition that $S_n = k$, but we never needed to find the probability of this event. It appeared as the denominator in the calculation of the posterior density of $X$:

$$ P(X \in dp \mid S_n = k) ~ = ~ \frac{C(r, s) p^{r-1}(1-p)^{s-1}dp \binom{n}{k} p^k (1-p)^{n-k}}{P(S_n = k)} $$

where $C(r, s)$ is the constant of integration in the beta $(r, s)$ density.

Because this results in the beta $(r+k, s+n-k)$ density we can equate constants to derive the identity

$$ \frac{C(r, s) \binom{n}{k}}{P(S_n = k)} ~ = ~ C(r+k, s+n-k) $$

So for $k$ in the range 0 through $n$,

$$ P(S_n = k) ~ = ~ \binom{n}{k} \frac{C(r, s)}{C(r+k, s+n-k)} $$

One $(r, s)$ pair is particularly interesting: $r = s = 1$. That's the case when $X$ has the uniform prior. The distribution of $S_n$ reduces to

$$ P(S_n = k ) ~ = ~ \frac{n!}{k!(n-k)!} \cdot \frac{1!}{0!0!} \cdot \frac{k!(n-k)!}{(n+1)!} ~ = ~ \frac{1}{n+1} $$

There's no $k$ in the answer! The conclusion is that if you choose $p$ uniformly between 0 and 1 and toss a $p$-coin $n$ times, the distribution of the number of heads is uniform on $\{ 0, 1, 2, \ldots, n\}$.

If you choose $p$ uniformly between 0 and 1, then for the conditional distribution of $S_n$ given that $p$ was the selected value is binomial $(n, p)$. But the unconditional distribution of $S_n$ is uniform.

Checking the Answer by Integration

If you prefer, you can find the distribution of $S_n$ directly, by conditioning on $X$.

\begin{align*} P(S_n = k) ~ &= \int_0^1 P(S_n = k \mid X = p)f_X(p)dp \\ \\ &= ~ \int_0^1 \binom{n}{k} p^k(1-p)^{n-k}C(r, s)p^{r-1}(1-p)^{s-1}dp \\ \\ &= ~ \binom{n}{k} C(r, s) \int_0^1 p^{r+k-1}(1-p)^{s+n-k-1} dp \\ \\ &= ~ \binom{n}{k} C(r, s) \frac{1}{C(r+k, s+n-k)} \end{align*}

Unconditional Expectation of $S_n$

Given $X = p$, the conditional distribution of $S_n$ is binomial $(n, p)$. Therefore

$$ E(S_n \mid X = p) ~ = ~ np $$

or, equivalently, $$ E(S_n \mid X) ~ = ~ nX $$ and so by iteration, $$ E(S_n) ~ = ~ E(nX) ~ = ~ nE(X) ~ = ~ n\frac{r}{r+s} $$

The expected proportion of heads is $$ E\big{(} \frac{S_n}{n} \big{)} ~ = ~ \frac{r}{r+s} $$

which is the expectation of the prior distribution of $X$.

In the next section we will examine the long run behavior of this random proportion.


The unconditional probability $P(S_n = k)$ appeared in the denominator of our calculation of the posterior density of $X$ given $S_n$. Because of the simplifications that result from using conjugate priors, we were able to calculate the denominator in a couple of different ways. But often the calculation can be intractable, especially in high dimensional settings. Methods of dealing with this problem are covered in more advanced courses.


results matching ""

    No results matching ""