Expectation

15.3. Expectation#

Let $X$ have density $f$ . Let $g$ be a real valued function on the real line, and suppose you want to find $E (g (X))$ . Then you can follow a procedure analogous to the non-linear function rule we developed for finding expectations of functions of discrete random variables.

Write a generic value of $X$ : that’s $x$ .
Apply the function $g$ to get $g (x)$ .
Weight $g (x)$ by the chance that $X$ is just around $x$ , resulting in the product $g (x) \cdot f (x) d x$ .
“Sum” over all $x$ , that is, integrate.

The expectation is

E (g (X)) = \int_{- \infty}^{\infty} g (x) \cdot f (x) d x

Technical Note: We must be careful here as $g$ is an arbitrary function and the integral above need not exist. If $g$ is non-negative, then the integral is either finite or diverges to $+ \infty$ , but it doesn’t oscillate. So if $g$ is non-negative, define

E (g (X)) = \int_{- \infty}^{\infty} g (x) \cdot f (x) d x provided the integral is finite.

For a general $g$ , first check whether $E (| g (X) |)$ is finite, that is, whether

\int_{- \infty}^{\infty} | g (x) | \cdot f (x) d x < \infty

If it is finite then there is a theorem that says $\int_{- \infty}^{\infty} g (x) \cdot f (x) d x$ exists, so it makes sense to define

E (g (X)) = \int_{- \infty}^{\infty} g (x) \cdot f (x) d x

Non-technical Note: In almost all of our examples, we will not be faced with questions about the existence of integrals. For example, if the set of possible values of $g (X)$ is bounded, then its expectation exists. But we will see a few examples of random variables that don’t have expectations. Such random variables have “heavy tails” and are important in many applications.

All the properties of means, variances, and covariances that we proved for discrete variables are still true. The proofs need to be rewritten for random variables with densities, but we won’t take the time to do that. Just use the properties as you did before. The Central Limit Theorem holds as well.

Quick Check

$X$ is a non-negative random variable that has density $f (x) = \frac{1}{120} x^{5} e^{- x}$ for $x \geq 0$ . Write a math expression for $E (\sqrt{X})$ . You don’t have to simplify it. [You can assume that $f$ is a density and that $E (\sqrt{X})$ is finite. We’ll check these facts later in the course.]

Answer

$E (\sqrt{X}) = \int_{0}^{\infty} x^{1 / 2} \cdot \frac{1}{120} x^{5} e^{- x} d x$

15.3.1. Uniform $(0, 1)$ #

The random variable $U$ is uniform on the unit interval if its density is flat over that interval and zero everywhere else:

\begin{array}{r} f_{U} (u) = {\begin{cases} 1 if 0 < u < 1 \\ 0 otherwise \end{cases} \end{array}

../../_images/c32bddc325e6aec7731f727b3dab9254d84c8552762b9173c47be198ed4e4beb.png

The area under $f_{U}$ over an interval is a rectangle. So it follows easily that the probability of an interval is its length relative to the total length of the unit interval, which is 1. For example, for every pair $u_{1}$ and $u_{2}$ with $u_{1} < u_{2}$ ,

P (u_{1} < U < u_{2}) = u_{2} - u_{1}

Equivalently, the cdf of $U$ is

\begin{array}{r} F_{U} (u) = {\begin{cases} 0 if u \leq 0 \\ u if 0 < u < 1 \\ 1 if u \geq 1 \end{cases} \end{array}

../../_images/0a8fd3de2be6bee7fd04c2aaaed2cadb019996be62b5272fca516b788965f18e.png

The expectation $E (U)$ doesn’t require an integral either. It’s the balance point of the density “curve”, which is 1/2. But if you insist, you can integrate:

E (U) = \int_{0}^{1} u \cdot 1 d u = \frac{1}{2}

For the variance, you do have to integrate. By the formula for expectation given at the start of this section,

E (U^{2}) = \int_{0}^{1} u^{2} \cdot 1 d u = \frac{1}{3} V a r (U) = \frac{1}{3} - (\frac{1}{2})^{2} = \frac{1}{12}

15.3.2. Uniform $(a, b)$ #

Fix $a < b$ . The uniform distribution on $(a, b)$ is flat over the interval $(a, b)$ and 0 elsewhere. Since its graph is a rectangle and the total area must be 1, the height of the rectangle is $\frac{1}{b - a}$ .

So if $X$ has the uniform $(a, b)$ distribution, then the density of $X$ is

f_{X} (x) = \frac{1}{b - a}, a < x < b

and 0 elsewhere. Probabilities are still relative lengths, so the cdf of $X$ is

F_{X} (x) = \frac{x - a}{b - a}, a < x < b

The expectation and variance of $X$ can be derived with little calculation once you notice that $X$ can be created by starting with a uniform $(0, 1)$ random variabe $U$ .

Step 1: $U$ is uniform on $(0, 1)$
Step 2: $(b - a) U$ is uniform on $(0, (b - a))$
Step 3: $X = a + (b - a) U$ is uniform on $(a, b)$ .

Now $X$ is a linear transformation of $U$ , so

E (X) = a + (b - a) E (U) = a + \frac{b - a}{2} = \frac{a + b}{2}

which is the midpoint of $(a, b)$ . Also,

V a r (X) = \frac{(b - a)^{2}}{12}

Quick Check

(a) Let $U$ be uniform on $(0, 1)$ . Find $c_{0}$ and $c_{1}$ such that $Y = c_{0} + c_{1} U$ is uniform on $(15, 20)$ .

(b) Let $X$ be uniform on $(- 2, 2)$ . Find $c_{0}$ and $c_{1}$ such that $V = c_{0} + c_{1} X$ is uniform on $(0, 1)$ .

Answer

(a) $c_{0} = 15$ , $c_{1} = 5$

(b) $c_{0} = 0.5$ , $c_{1} = 0.25$

15.3.3. Example: Random Discs#

A screen saver chooses a random radius uniformly in the interval $(0, 2)$ centimeters and draws a disc with that radius. Then it chooses another radius in the same way, independently of the first, and draws another disc. And so on.

Question 1. Let $S$ be the area of the first disc. Find $E (S)$ .

Answer. Let $R$ be the radius of the first disc. Then $S = π R^{2}$ . So

E (S) = π E (R^{2}) = π (V a r (R) + (E (R))^{2}) = π (\frac{4}{12} + 1^{2}) = 4.19 c m^{2}

np.pi * (4/12 + 1)

4.1887902047863905

Question 2. Let $\bar{R}$ be the average radius of the first 100 discs. Find a number $c$ so that $P (| \bar{R} - 1 | < c) \approx 99 %$ .

Answer. Let $R_{1}, R_{2}, \dots, R_{100}$ be the first 100 radii. These are i.i.d. random variables, each with mean 1 and variance $4 / 12$ . So $E (\bar{R}) = 1$ and

S D (\bar{R}) = \frac{\sqrt{4 / 12}}{\sqrt{100}} = 0.0577 cm

sd_rbar = ((4/12)**0.5)/(100**0.5)
sd_rbar

0.057735026918962574

By the Central Limit Theorem, the distribution of $\bar{R}$ is approximately normal. Let’s draw it using Plot_norm.

Plot_norm((0.8, 1.2), 1, sd_rbar)
plt.xlabel('Radius in Centimeters')
plt.title('Approximate Distribution of Sample Mean Radius');

../../_images/df83b1704c30dcb46124d32cdef07c0f4992e53b3bfcd2e3e38f878dc590c63e.png

We are looking for $c$ such that there is about 99% chance that $\bar{R}$ is in the interval $(1 - c, 1 + c)$ . Therefore $1 + c$ is the 99.5th (not 99th) percent point of the curve above, from which you can find $c$ .

z = stats.norm.ppf(0.995)
z

2.5758293035489004

c = z*sd_rbar
c

0.14871557417904838

We can now get the endpoints of the interval. The graph below shows the corresponding area of 99%.

1-c, 1+c

(0.85128442582095165, 1.1487155741790485)

Plot_norm((0.8, 1.2), 1, sd_rbar, left_end = 1-c, right_end = 1+c)
plt.xticks([1-c, 1, 1+c])
plt.xlabel('Radius in Centimeters')
plt.title('Gold Area is Approximately 99%');

../../_images/bd3ada5eccd318ca71f2c8f1f69185dcb1e0f4740aca44ba9e13e2c9f15d364a.png

Quick Check

Let $X_{1}, X_{2}, \dots, X_{100}$ be i.i.d. uniform on $(0, 1)$ , and let $A_{n} = \frac{1}{n} \sum_{i = 1}^{n} X_{i}$ . Find or approximate the distribution of $A_{n}$ . If it’s a famous one, provide its name and parameters.

Answer

Roughly normal, mean $1 / 2$ , variance $1 / 1200$

Expectation

Contents

15.3. Expectation#

15.3.1. Uniform (0,1)#

15.3.2. Uniform (a,b)#

15.3.3. Example: Random Discs#

15.3.1. Uniform $(0, 1)$ #

15.3.2. Uniform $(a, b)$ #