# 18.4. Chi-Squared Distributions¶

The gamma family has two important branches. The first consists of gamma $$(r, \lambda)$$ distributions with integer shape parameter $$r$$, as you saw in the previous section.

The other important branch consists of gamma $$(r, \lambda)$$ distributions that have half-integer shape parameter $$r$$, that is, when $$r = n/2$$ for a positive integer $$n$$. Notice that this branch contains the one above: every integer $$r$$ is also half of the integer $$n = 2r$$.

## 18.4.1. Chi-Squared $$(1)$$¶

We have already seen the fundamental member of the branch. Let $$Z$$ be a standard normal random variable and let $$V = Z^2$$. By the change of variable formula for densities, we found the density of $$V$$ to be

$f_V(v) ~ = ~ \frac{1}{\sqrt{2\pi}} v^{-\frac{1}{2}} e^{-\frac{1}{2} v}, ~~~~ v > 0$

That’s the gamma $$(1/2, 1/2)$$ density. It is also called the chi-squared density with 1 degree of freedom, which we will abbreviate to chi-squared (1).

## 18.4.2. From Chi-Squared $$(1)$$ to Chi-Squared $$(n)$$¶

When we were establishing the properties of the standard normal density, we discovered that if $$Z_1$$ and $$Z_2$$ are independent standard normal then $$Z_1^2 + Z_2^2$$ has the exponential $$(1/2)$$ distribution. We saw this by comparing two different settings in which the Rayleigh distribution arises. But that wasn’t a particularly illuminating reason for why $$Z_1^2 + Z_2^2$$ should be exponential.

But now we know that the sum of independent gamma variables with the same rate is also gamma; the shape parameter adds up and the rate remains the same. Therefore $$Z_1^2 + Z_2^2$$ is a gamma $$(1, 1/2)$$ variable. That’s the same distribution as exponential $$(1/2)$$, as you showed in exercises. This explains why the sum of squares of two i.i.d. standard normal variables has the exponential $$(1/2)$$ distribution.

If $$Z_1, Z_2, Z_3$$ are i.i.d. standard normal variables, then:

• $$Z_1^2$$ has the gamma $$(1/2, 1/2)$$ distribution

• $$Z_1^2 + Z_2^2$$ has the gamma $$(1/2 + 1/2, 1/2)$$ distribution

• $$Z_1^2 + Z_2^2 + Z_3^2$$ has the gamma $$(1/2 + 1/2 + 1/2, 1/2)$$ distribution

Now let $$Z_1, Z_2, \ldots, Z_n$$ be i.i.d. standard normal variables. Then $$Z_1^2, Z_2^2, \ldots, Z_n^2$$ are i.i.d. chi-squared $$(1)$$ variables. That is, each of them has the gamma $$(1/2, 1/2)$$ distribution.

By induction, $$Z_1^2 + Z_2^2 + \cdots + Z_n^2$$ has the gamma $$(n/2, 1/2)$$ distribution. This is called the chi-squared distribution with $$n$$ degrees of freedom, which we will abbreviate to chi-squared $$(n)$$.

In data science, these distributions often arise when we work with the sum of squares of normal errors. This is usually part of a mean squared error calculation.

## 18.4.3. Chi-Squared Distribution with $$n$$ Degrees of Freedom¶

For a positive integer $$n$$, the random variable $$X$$ has the chi-squared distribution with $$n$$ degrees of freedom if the distribution of $$X$$ is gamma $$(n/2, 1/2)$$. That is, $$X$$ has density

$f_X(x) ~ = ~ \frac{\frac{1}{2}^{\frac{n}{2}}}{\Gamma(\frac{n}{2})} x^{\frac{n}{2} - 1} e^{-\frac{1}{2}x}, ~~~~ x > 0$

Here are the graphs of the chi-squared densities for degrees of freedom 2 through 5. The chi-squared (2) distribution is exponential because it is the gamma $$(1, 1/2)$$ distribution. This distribution has three names:

• chi-squared (2)

• gamma (1, 1/2)

• exponential (1/2)

## 18.4.4. Mean and Variance¶

You know that if $$T$$ has the gamma $$(r, \lambda)$$ density then

$E(T) ~ = ~ \frac{r}{\lambda} ~~~~~~~~~~~~ SD(T) = \frac{\sqrt{r}}{\lambda}$

If $$X$$ has the chi-squared $$(n)$$ distribution then $$X$$ is gamma $$(n/2, 1/2)$$. So

$E(X) ~ = ~ \frac{n/2}{1/2} ~ = ~ n$

Thus the expectation of a chi-squared random variable is its degrees of freedom.

The SD is $$SD(X) ~ = ~ \frac{\sqrt{n/2}}{1/2} ~ = ~ \sqrt{2n}$$

## 18.4.5. Estimating the Normal Variance¶

Suppose $$X_1, X_2, \ldots, X_n$$ are i.i.d. normal $$(\mu, \sigma^2)$$ variables, and that you are in a setting in which you know $$\mu$$ and are trying to estimate $$\sigma^2$$.

Let $$Z_i$$ be $$X_i$$ in standard units, so that $$Z_i = (X_i - \mu)/\sigma$$. Define the random variable $$T$$ as follows:

$T ~ = ~ \sum_{i=1}^n Z_i^2 ~ = ~ \frac{1}{\sigma^2}\sum_{i=1}^n (X_i - \mu)^2$

Then $$T$$ has the chi-squared $$(n)$$ distribution and $$E(T) = n$$. Now define $$W$$ by

$W ~ = ~ \frac{\sigma^2}{n} T ~ = ~ \frac{1}{n} \sum_{i=1}^n (X_i - \mu)^2$

Then $$W$$ can be computed based on the sample since $$\mu$$ is known. And since $$W$$ is a linear tranformation of $$T$$ it is easy to see that $$E(W) = \sigma^2$$.

So we have constructed an unbiased estimate of $$\sigma^2$$. It is the mean squared deviation from the known population mean.

But typically, $$\mu$$ is not known. In that case you need a different estimate of $$\sigma^2$$ since you can’t compute $$W$$ as defined above. You showed in exercises that

$S^2 ~ = ~ \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X})^2$

is an unbiased estimate of $$\sigma^2$$ regardless of the distribution of the $$X_i$$’s. When the $$X_i$$’s are normal, as is the case here, it turns out that $$S^2$$ is a linear transformation of a chi-squared $$(n-1)$$ random variable. We will show that later in the course.

## 18.4.6. “Degrees of Freedom”¶

The example above helps explain the strange term “degrees of freedom” for the parameter of the chi-squared distribution.

• When $$\mu$$ is known, you have $$n$$ independent centered normals $$(X_i - \mu)$$ that you can use to estimate $$\sigma^2$$. That is, you have $$n$$ degrees of freedom in constructing your estimate.

• When $$\mu$$ is not known, you are using all $$n$$ of $$X_1 - \bar{X}, X_2 - \bar{X}, \ldots, X_n - \bar{X}$$ in your estimate, but they are not independent. They are the deviations of the list $$X_1, X_2, \ldots , X_n$$ from their average $$\bar{X}$$, and hence their sum is 0. If you know $$n-1$$ of them, the final one is determined. So you only have $$n-1$$ degrees of freedom.