Day 3: Random Variables, Expectation & Variance

Theory

Random variables

A random variable $X$ is a function from the sample space to the real numbers, $X: \Omega \to \mathbb{R}$ . It turns outcomes ("the firm adopted clean tech") into numbers (emissions in tonnes). Its distribution is described by a probability mass function $p(x) = P(X = x)$ (discrete) or a density $f(x)$ (continuous), with $P(a \le X \le b) = \int_a^b f(x)\,dx$ .

Expectation

The expected value is the probability-weighted average:

E[X] = \sum_x x\,p(x) \qquad\text{or}\qquad E[X] = \int_{-\infty}^{\infty} x f(x)\,dx.

For any function $g$ , $E[g(X)] = \sum_x g(x)p(x)$ (the "law of the unconscious statistician").

Linearity of expectation holds for any random variables, dependent or not:

E[aX + bY + c] = aE[X] + bE[Y] + c.

Variance

\operatorname{Var}(X) = E\big[(X - E[X])^2\big] = E[X^2] - (E[X])^2, \qquad \operatorname{Var}(aX + b) = a^2 \operatorname{Var}(X).

The standard deviation $\operatorname{sd}(X) = \sqrt{\operatorname{Var}(X)}$ is in the units of $X$ . Shifting a variable moves its centre but not its spread; scaling by $a$ scales the spread by $|a|$ .

Jensen's inequality

If $g$ is convex, $E[g(X)] \ge g(E[X])$ ; if concave, the inequality flips. In general $E[g(X)] \ne g(E[X])$ , which is why the average of logs is not the log of the average (Day 12 builds on this).

Potential outcomes as random variables

Each unit has two potential outcomes: $Y(1)$ if treated and $Y(0)$ if not. The average treatment effect is an expectation:

\tau_{ATE} = E[Y(1) - Y(0)] = E[Y(1)] - E[Y(0)],

where the second equality is linearity. We will never observe $Y(1) - Y(0)$ for anyone, but linearity tells us we only need the two means separately.

Part 1 · Warm-up

Exercise 1 · Expected fine

A random inspection yields a fine of $0 with probability 0.6, $500 with probability 0.3 and $2,000 with probability 0.1. Find $E[X]$ .

Show solution

E[X] = 0(0.6) + 500(0.3) + 2000(0.1) = 0 + 150 + 200 = 350.

Exercise 2 · A second moment

For the same fine, compute $E[X^2]$ .

Show solution

E[X^2] = 0^2(0.6) + 500^2(0.3) + 2000^2(0.1) = 75{,}000 + 400{,}000 = 475{,}000.

Exercise 3 · Variance by the shortcut

Compute $\operatorname{Var}(X)$ and $\operatorname{sd}(X)$ .

Show solution

\operatorname{Var}(X) = 475{,}000 - 350^2 = 475{,}000 - 122{,}500 = 352{,}500,

so $\operatorname{sd}(X) = \sqrt{352{,}500} \approx 593.7$ dollars.

Exercise 4 · Linearity

Tax revenue is $Y = 15X - 100$ , where $X$ is emissions with $E[X] = 1{,}200$ tonnes. Find $E[Y]$ .

Show solution

$E[Y] = 15(1200) - 100 = 17{,}900$ .

Exercise 5 · Variance of a linear transformation

If $\operatorname{Var}(X) = 400$ , what are $\operatorname{Var}(Y)$ and $\operatorname{sd}(Y)$ ?

Show solution

Step 1. Use $\operatorname{Var}(aX + b) = a^2\operatorname{Var}(X)$ with $a = 15$ , $b = -100$ :

\operatorname{Var}(Y) = 15^2 \times 400 = 225 \times 400 = 90{,}000.

Step 2. Standard deviation: $\operatorname{sd}(Y) = |15| \times \sqrt{400} = 15 \times 20 = 300$ .

Answer: variance 90,000 and sd 300. The constant $-100$ shifts revenue but does not change its spread.

Part 2 · Proofs

Exercise 6 · Linearity with constants

Prove $E[aX + b] = aE[X] + b$ for a discrete $X$ .

Show solution

E[aX + b] = \sum_x (ax + b)p(x) = a\sum_x x\,p(x) + b\sum_x p(x) = aE[X] + b,

using $\sum_x p(x) = 1$ .

Exercise 7 · The variance shortcut

Prove $\operatorname{Var}(X) = E[X^2] - (E[X])^2$ .

Show solution

Let $\mu = E[X]$ . Expand the square and use linearity:

E[(X - \mu)^2] = E[X^2] - 2\mu E[X] + \mu^2 = E[X^2] - 2\mu^2 + \mu^2 = E[X^2] - \mu^2.

Exercise 8 · Variance scaling

Prove $\operatorname{Var}(aX + b) = a^2\operatorname{Var}(X)$ .

Show solution

$E[aX + b] = a\mu + b$ , so

\operatorname{Var}(aX + b) = E\big[(aX + b - a\mu - b)^2\big] = E\big[a^2 (X - \mu)^2\big] = a^2\operatorname{Var}(X).

Exercise 9 · Indicator variables

Let $\mathbf{1}_A$ equal 1 if $A$ happens and 0 otherwise. Show that $E[\mathbf{1}_A] = P(A)$ and $\operatorname{Var}(\mathbf{1}_A) = P(A)(1 - P(A))$ .

Show solution

Step 1. The indicator takes the value 1 with probability $P(A)$ and 0 otherwise:

E[\mathbf{1}_A] = 1 \cdot P(A) + 0 \cdot P(A^c) = P(A).

Step 2. Since $1^2 = 1$ and $0^2 = 0$ , $\mathbf{1}_A^2 = \mathbf{1}_A$ , so $E[\mathbf{1}_A^2] = P(A)$ .

Step 3. Variance shortcut:

\operatorname{Var}(\mathbf{1}_A) = E[\mathbf{1}_A^2] - (E[\mathbf{1}_A])^2 = P(A) - P(A)^2 = P(A)(1 - P(A)).

This is why a share (e.g. "share treated") is simply the mean of a 0/1 variable.

Exercise 10 · The mean is the best constant prediction

Show that $E[(X - c)^2] = \operatorname{Var}(X) + (E[X] - c)^2$ , and conclude that $c = E[X]$ minimises the mean squared error.

Show solution

Step 1. Write $X - c = (X - \mu) + (\mu - c)$ and expand:

E[(X - c)^2] = E[(X - \mu)^2] + 2(\mu - c)\underbrace{E[X - \mu]}_{=0} + (\mu - c)^2 = \operatorname{Var}(X) + (\mu - c)^2.

Step 2. The second term is non-negative and zero only at $c = \mu$ .

Step 3. This is the seed of least squares: regression generalises "predict with the mean" to "predict with a function of $X$ " (Day 6).

Part 3 · Applied

Exercise 11 · An average treatment effect

A (hypothetical) table shows both potential outcomes for four workers:

Worker	$Y(1)$	$Y(0)$
1	10	7
2	8	8
3	12	9
4	6	4

Compute each individual effect, the ATE, and check that $\text{ATE} = E[Y(1)] - E[Y(0)]$ .

Show solution

Step 1. Individual effects $Y(1) - Y(0)$ : $10 - 7 = 3$ , $8 - 8 = 0$ , $12 - 9 = 3$ , $6 - 4 = 2$ .

Step 2. ATE = average of the individual effects: $(3 + 0 + 3 + 2)/4 = 2$ .

Step 3. Means of each potential outcome: $E[Y(1)] = (10 + 8 + 12 + 6)/4 = 9$ and $E[Y(0)] = (7 + 8 + 9 + 4)/4 = 7$ .

Step 4. Difference of means: $9 - 7 = 2$ . ✓

Answer: ATE = 2 either way, as linearity of expectation promises.

Exercise 12 · How much to treat

A pilot cash-transfer programme assigns each eligible household to treatment with probability $p$ , so $T \sim \text{Bernoulli}(p)$ . Compute $\operatorname{Var}(T)$ when $p = 0.3$ and find the $p$ that maximises it. Why might the evaluation team care?

Show solution

Step 1. By Exercise 9, $\operatorname{Var}(T) = p(1 - p)$ . At $p = 0.3$ : $0.3 \times 0.7 = 0.21$ .

Step 2. Maximise $g(p) = p - p^2$ : $g'(p) = 1 - 2p = 0 \Rightarrow p = 0.5$ , and $g''(p) = -2 < 0$ , so it is a maximum.

Step 3. Maximum value: $0.5 \times 0.5 = 0.25$ .

Answer: 0.21 at $p = 0.3$ ; maximised at $p = 0.5$ (value 0.25). More variation in treatment means more information about its effect, so a 50/50 split gives the most precise estimate for a given number of households (Day 10).

Exercise 13 · A continuous voucher

A voucher amount is uniform on $[0, 10]$ (thousand dollars), with density $f(x) = 1/10$ . Compute $E[X]$ and $\operatorname{Var}(X)$ by integration.

Show solution

The density is $f(x) = 1/10$ on $[0, 10]$ .

Step 1. Mean:

E[X] = \int_0^{10} x\cdot\frac{1}{10}\,dx = \frac{1}{10}\Big[\frac{x^2}{2}\Big]_0^{10} = \frac{1}{10}\cdot 50 = 5.

Step 2. Second moment:

E[X^2] = \int_0^{10} x^2\cdot\frac{1}{10}\,dx = \frac{1}{10}\Big[\frac{x^3}{3}\Big]_0^{10} = \frac{1000}{30} \approx 33.33.

Step 3. Variance: $\operatorname{Var}(X) = 33.33 - 5^2 = 8.33$ .

Answer: mean $5k and variance 8.33 (sd ≈ $2.9k), matching the formula $(b - a)^2/12 = 100/12$ .

Exercise 14 · Risk aversion and Jensen

A household's income is $50k or $150k with equal probability, and its utility is $u(y) = \ln y$ . Compare $E[\ln Y]$ with $\ln E[Y]$ and find the certainty equivalent.

Show solution

Step 1. Expected utility:

E[\ln Y] = \tfrac12\ln 50 + \tfrac12\ln 150 \approx \tfrac12(3.9120 + 5.0106) = 4.4613.

Step 2. Utility of the expected income: $E[Y] = 100$ , so $\ln E[Y] = \ln 100 \approx 4.6052$ .

Step 3. Compare: $4.4613 < 4.6052$ , as Jensen's inequality predicts for the concave $\ln$ .

Step 4. The certainty equivalent $c$ gives the same utility as the gamble: $\ln c = 4.4613$ , so

c = e^{4.4613} = \sqrt{50 \times 150} \approx 86.6.

Answer: the household values the gamble like a sure $86.6k, although it is worth $100k on average. The $13.4k gap is the risk premium.

Exercise 15 · Effect heterogeneity is not identified

For the four workers in Exercise 11, compute $\operatorname{Var}(Y(1) - Y(0))$ . Could you compute it from real data, where each worker shows only one potential outcome?

Show solution

Step 1. Individual effects from Exercise 11: 3, 0, 3, 2, with mean 2.

Step 2. Deviations from the mean: 1, −2, 1, 0; squared: 1, 4, 1, 0.

Step 3. Population variance: $(1 + 4 + 1 + 0)/4 = 1.5$ .

Step 4. Could real data give this? Expand the variance of a difference:

\operatorname{Var}(Y(1) - Y(0)) = \operatorname{Var}(Y(1)) + \operatorname{Var}(Y(0)) - 2\operatorname{Cov}(Y(1), Y(0)).

The two variances can be estimated from the treated and control groups, but the covariance needs both outcomes for the same worker, which we never see.

Answer: 1.5 in this table, but in real data the variance of effects is not identified. Means are identified by linearity; the spread of effects is not.

Verify in Python

import numpy as np

x = np.array([0, 500, 2000]); p = np.array([0.6, 0.3, 0.1])
ex, ex2 = (x * p).sum(), (x**2 * p).sum()
print("E[X] =", ex, " E[X^2] =", ex2, " Var =", ex2 - ex**2)

y1 = np.array([10, 8, 12, 6]); y0 = np.array([7, 8, 9, 4])
print("ATE =", (y1 - y0).mean(), "  Var of effects =", (y1 - y0).var())

# Jensen: average of logs < log of average
inc = np.array([50, 150])
print("E[ln Y] =", np.log(inc).mean().round(4), " ln E[Y] =", np.log(inc.mean()).round(4),
      " certainty equivalent =", np.exp(np.log(inc).mean()).round(2))

# Linearity check by simulation: Y = 15X - 100 with Var(X) = 400
rng = np.random.default_rng(0)
X = rng.normal(1200, 20, 1_000_000); Y = 15 * X - 100
print("E[Y] ~", Y.mean().round(1), " Var(Y) ~", Y.var().round(0))

Key takeaways

Expectation is linear for any random variables; variance scales with the square of a multiplier and ignores shifts.
The ATE is a difference of two means, so it can be identified even though no individual effect is observed.
Non-linear functions do not commute with expectation (Jensen).

Day 3. Random Variables, Expectation & Variance

Theory

Random variables

Expectation

Variance

Jensen's inequality

Potential outcomes as random variables

Part 1 · Warm-up

Exercise 1 · Expected fine

Exercise 2 · A second moment

Exercise 3 · Variance by the shortcut

Exercise 4 · Linearity

Exercise 5 · Variance of a linear transformation

Part 2 · Proofs

Exercise 6 · Linearity with constants

Exercise 7 · The variance shortcut

Exercise 8 · Variance scaling

Exercise 9 · Indicator variables

Exercise 10 · The mean is the best constant prediction

Part 3 · Applied

Exercise 11 · An average treatment effect

Exercise 12 · How much to treat

Exercise 13 · A continuous voucher

Exercise 14 · Risk aversion and Jensen

Exercise 15 · Effect heterogeneity is not identified

Verify in Python

Key takeaways

Simulation Lab: Linearity of Expectation & Variance Scaling