Expectations of Functions

8.3. Expectations of Functions#

Once we have a random variable, we often want to work with functions of it. For example, if a random variables is an estimator, we usually want to see how far it is from the value it is trying to estimate. For example, we might want to see how far a random variable $X$ is from the number 10. That’s a function of $X$ . Let’s call it $Y$ . Then

Y = | X - 10 |

This section is about finding the expectation of a function of a random variable whose distribution you know. Throughout, we will assume that all the expectations that we are discussing are well defined.

In what follows, let $X$ be a random variable whose distribution (and hence also expectation) are known.

8.3.1. Linear Function Rule#

Let $X$ be a random variable with expectation $E (X)$ and let $Y = a X + b$ for some constants $a$ and $b$ .

This kind of transformation happens for example when you change units of measurement.

If you switch from Celsius to Fahreneheit, then $a = 9 / 5$ and $b = 32$ .
If you switch from inches to centimeters, then $a = 2.54$ and $b = 0$ .

We can find $E (Y)$ by applying the definition of expectation on the domain $Ω$ . For every $ω \in Ω$ , we have $Y (ω) = a X (ω) + b$ . So

\begin{array}{r} \begin{aligned} E (Y) & = \sum_{all ω} (a X (ω) + b) P (ω) \\ = a \sum_{all ω} X (ω) P (ω) + b \sum_{all ω} P (ω) \\ = a E (X) + b \end{aligned} \end{array}

For example, $E (2 X - 3) = 2 E (X) - 3$ . Also $E (X / 2) = E (X) / 2$ , and $E (1 - X) = 1 - E (X)$ .

The expectation of a linear transformation of $X$ is the linear transformation of the expectation of $X$ . This is a handy result as we will often be transforming variables linearly.

Quick Check

Let $X$ be a random variable with $E (X) = 5$ .

Find $E (20 + 2 X - (10 - X))$ .

Answer

$25$

But expectation behaves differently under non-linear transformation.

8.3.2. Non-linear Function Rule#

Now let $Y = g (X)$ where $g$ is any numerical function. Remember that $X$ is a function on $Ω$ . So the function that defines the random variable $Y$ is a composition:

Y (ω) = (g \circ X) (ω) for ω \in Ω

This allows us to write $E (Y)$ in three equivalent ways:

On the range of $Y$

E (Y) = \sum_{all y} y P (Y = y)

On the domain $Ω$

E (Y) = E (g (X)) = \sum_{ω \in Ω} (g \circ X) (ω) P (ω)

On the range of $X$

E (Y) = E (g (X)) = \sum_{all x} g (x) P (X = x)

As before, it is a straightforward matter of grouping to show that all the forms are equivalent.

The first form looks the simplest, but there’s a catch: you need to first find $P (Y = y)$ . The second form involves an unnecessarily high level of detail.

The third form is the one to use. It uses the known distribution of $X$ . It says that to find $E (Y)$ where $Y = g (X)$ for some function $g$ :

Take a generic value $x$ of $X$ .
Apply $g$ to $x$ ; this $g (x)$ is a generic value of $Y$ .
Weight $g (x)$ by $P (X = x)$ , which is known.
Do this for all $x$ and add. The sum is $E (Y)$ .

The crucial thing to note about this method is that we didn’t have to first find the distribution of $Y$ . That saves us a lot of work.

Let’s see how our method works in some examples.

8.3.3. $Y = | X - 3 |$ #

Let $X$ have a distribution we worked with earlier:

x = np.arange(1, 6)
probs = make_array(0.15, 0.25, 0.3, 0.2, 0.1)
dist = Table().values(x).probabilities(probs)
dist = dist.relabel('Value', 'x').relabel('Probability', 'P(X=x)')
dist

x	P(X=x)
1	0.15
2	0.25
3	0.3
4	0.2
5	0.1

Let $g$ be the function defined by $g (x) = | x - 3 |$ , and let $Y = g (X)$ . In other words, $Y = | X - 3 |$ .

To calculate $E (Y)$ , we first have to create a column that transforms the values of $X$ into values of $Y$ :

dist_with_Y = dist.with_column('g(x)', np.abs(dist.column('x')-3)).move_to_end('P(X=x)')

dist_with_Y

x	g(x)	P(X=x)
1	2	0.15
2	1	0.25
3	0	0.3
4	1	0.2
5	2	0.1

To get $E (Y)$ , find the appropriate weighed average: multiply the g(x) and P(X=x) columns, and add. The calculation shows that $E (Y) = 0.95$ .

ev_Y = sum(dist_with_Y.column('g(x)') * dist_with_Y.column('P(X=x)'))
ev_Y

0.94999999999999996

8.3.4. $Y = min (X, 3)$ #

Let $X$ be as above, but now let $Y = min (X, 3)$ . We want $E (Y)$ . What we know is the distribution of $X$ :

dist

x	P(X=x)
1	0.15
2	0.25
3	0.3
4	0.2
5	0.1

To find $E (Y)$ we can just go row by row and replace the value of $x$ by the value of $min (x, 3)$ , and then find the weighted average:

ev_Y = 1*0.15 + 2*0.25 + 3*0.3 + 3*0.2 + 3*0.1
ev_Y

2.45

8.3.5. $E (X (X - 1))$ for a Poisson Variable $X$ #

Let $X$ have the Poisson $(μ)$ distribution.

\begin{array}{r} \begin{aligned} E (X (X - 1)) & = \sum_{k = 0}^{\infty} k (k - 1) e^{- μ} \frac{μ^{k}}{k!} \\ = e^{- μ} μ^{2} \sum_{k = 2}^{\infty} \frac{μ^{k - 2}}{(k - 2)!} \\ = e^{- μ} μ^{2} e^{μ} \\ = μ^{2} \end{aligned} \end{array}

In the next section we will use this to find $E (X^{2})$ . For now, notice that

E (X^{2}) = \sum_{k = 0}^{\infty} k^{2} e^{- μ} \frac{μ^{k}}{k!} \geq \sum_{k = 0}^{\infty} k (k - 1) e^{- μ} \frac{μ^{k}}{k!} = E (X (X - 1)) = μ^{2}

Since $E (X) = μ$ , we have $E (X^{2}) \geq (E (X))^{2}$ . We will see later that this inequality is true for all random variables for which the expected square is finite.

Expectations of Functions

Contents