Day 14: Missing Data & Inverse Probability Weighting

Theory

The response indicator

Let $R = 1$ if $Y$ is observed and $R = 0$ if it is missing; $X$ is always observed. The mechanism is the distribution of $R$ given the data:

Mechanism	Definition	Complete-case mean
MCAR (missing completely at random)	$P(R = 1 \mid X, Y) = P(R = 1)$	unbiased
MAR (missing at random)	$P(R = 1 \mid X, Y) = P(R = 1 \mid X)$	biased in general; fixable with $X$
MNAR (missing not at random)	depends on $Y$ even given $X$	biased; needs extra assumptions

Inverse probability weighting (IPW)

Under MAR with response propensity $\pi(X) = P(R = 1 \mid X) > 0$ , weighting each observed unit by $1/\pi(X)$ rebuilds the full population:

E\Big[\frac{R\,Y}{\pi(X)}\Big] = E[Y].

Two sample versions:

\hat\mu_{HT} = \frac1n\sum_{i=1}^n \frac{R_iY_i}{\pi(X_i)}, \qquad \hat\mu_{\text{Hájek}} = \frac{\sum_i R_iY_i/\pi(X_i)}{\sum_i R_i/\pi(X_i)}.

The Hájek (normalised) version is usually more stable.

Multiple imputation and Rubin's rules

Impute the missing values $m$ times from a model, estimate $\hat Q_j$ (with variance $W_j$ ) in each completed data set, then combine:

\bar Q = \frac1m\sum_j\hat Q_j, \qquad T = \bar W + \Big(1 + \frac1m\Big)B, \qquad B = \frac{1}{m - 1}\sum_j(\hat Q_j - \bar Q)^2.

The between-imputation part $B$ reflects uncertainty about the missing values themselves.

Causal inference is a missing-data problem

Each unit shows $Y(1)$ if treated and $Y(0)$ if not; the other is missing. Randomisation makes the missing potential outcome MCAR; unconfoundedness given $X$ makes it MAR; and IPW with the propensity score $e(X) = P(T = 1 \mid X)$ is exactly today's estimator, applied twice. Phase 3 builds on this.

Part 1 · Warm-up

Exercise 1 · Spot the mechanism (1)

A lab's freezer fails and destroys a random 10% of blood samples. Which mechanism?

Show solution

MCAR: missingness is unrelated to anything about the patients.

Exercise 2 · Spot the mechanism (2)

High earners are more likely to skip the income question, even among people with the same education and age. Which mechanism?

Show solution

MNAR: missingness depends on the missing value (income) itself, even after conditioning on observed covariates.

Exercise 3 · Spot the mechanism (3)

Older respondents are less likely to answer an online follow-up, but, within each age group, response is unrelated to the outcome. Which mechanism?

Show solution

MAR: missingness depends only on the observed covariate (age).

Exercise 4 · An IPW weight

A respondent's group has a response rate of 25%. What weight does each respondent get, and why?

Show solution

$1/0.25 = 4$ : each respondent stands in for themselves and three similar non-respondents.

Exercise 5 · Complete cases under MCAR

Why is the complete-case mean unbiased under MCAR? What is lost?

Show solution

Respondents are a random subsample, so their mean has the same expectation as the full-sample mean. The cost is precision: a smaller $n$ means a larger standard error.

Part 2 · Proofs

Exercise 6 · Complete-case mean under MCAR

Show that $E[Y \mid R = 1] = E[Y]$ under MCAR.

Show solution

MCAR means $R$ is independent of $Y$ , so $E[Y \mid R = 1] = E[Y]$ (Day 6, Exercise 5).

Exercise 7 · IPW is unbiased under MAR

Prove $E[RY/\pi(X)] = E[Y]$ when $P(R = 1 \mid X, Y) = \pi(X) > 0$ .

Show solution

Condition on $(X, Y)$ and use the LIE:

E\Big[\frac{RY}{\pi(X)}\Big] = E\Big[\frac{Y}{\pi(X)}E[R \mid X, Y]\Big] = E\Big[\frac{Y}{\pi(X)}\pi(X)\Big] = E[Y].

The step $E[R \mid X, Y] = \pi(X)$ is the MAR assumption, and $\pi(X) > 0$ (overlap) is needed to divide.

Exercise 8 · The bias under MNAR

Show that $E[Y \mid R = 1] - E[Y] = \operatorname{Cov}(R, Y)/P(R = 1)$ .

Show solution

Step 1. $E[RY] = P(R = 1)E[Y \mid R = 1]$ , so $\operatorname{Cov}(R, Y) = P(R = 1)E[Y \mid R = 1] - P(R = 1)E[Y]$ .

Step 2. Divide by $P(R = 1)$ .

Step 3. If response rises with $Y$ , the covariance is positive and respondents overstate the mean.

Exercise 9 · Mean imputation understates variance

Fill $n - r$ missing values with the mean of the $r$ observed values. Show that the sample variance of the completed data is $\frac{r - 1}{n - 1}s_r^2$ , where $s_r^2$ is the observed-data variance.

Show solution

Step 1. Imputed values equal the observed mean $\bar y_r$ , so the completed-data mean is still $\bar y_r$ , and they add zero to the sum of squares.

Step 2. That sum stays $(r - 1)s_r^2$ but is now divided by $n - 1$ .

Step 3. With 30% missing, the variance shrinks by about 30% and standard errors are too small.

Exercise 10 · Rubin's rules

Five imputations give estimates 10.2, 9.8, 10.5, 10.0 and 9.5, each with within-imputation variance 0.25. Find the pooled estimate and its SE.

Show solution

$\bar Q = 50/5 = 10.0$ . Deviations 0.2, −0.2, 0.5, 0, −0.5 give $B = 0.58/4 = 0.145$ . Then

T = 0.25 + (1 + 1/5)(0.145) = 0.25 + 0.174 = 0.424, \qquad \operatorname{SE} = \sqrt{0.424} \approx 0.651.

Part 3 · Applied

Exercise 11 · Bias from MNAR

30% of people are high earners ( $Y = 100$ ) and 70% low earners ( $Y = 40$ ). High earners respond with probability 0.5, low earners with 0.9. Find the true mean and the respondents' mean.

Show solution

Step 1. True mean: $0.3(100) + 0.7(40) = 30 + 28 = 58$ .

Step 2. Share of the population who respond: high earners $0.3 \times 0.5 = 0.15$ ; low earners $0.7 \times 0.9 = 0.63$ ; total 0.78.

Step 3. Respondents' mean:

\frac{0.15(100) + 0.63(40)}{0.78} = \frac{15 + 25.2}{0.78} \approx 51.5.

Answer: the survey reports 51.5 instead of 58, a bias of about −6.5, because high earners are under-represented among respondents.

Exercise 12 · Repairing MAR with weights

Half the sample is young (mean $Y = 20$ , response rate 0.4) and half old (mean $Y = 40$ , response rate 0.8); within age, response is random. Compare the complete-case mean with the IPW mean.

Show solution

Step 1. True mean: $0.5(20) + 0.5(40) = 30$ .

Step 2. Respondents: young $0.5 \times 0.4 = 0.2$ , old $0.5 \times 0.8 = 0.4$ (total 0.6).

Step 3. Complete-case mean: $(0.2 \times 20 + 0.4 \times 40)/0.6 = 20/0.6 \approx 33.3$ (old people over-represented).

Step 4. IPW weights: young $1/0.4 = 2.5$ , old $1/0.8 = 1.25$ . Weighted mean per person in the population:

0.2 \times 2.5 \times 20 + 0.4 \times 1.25 \times 40 = 10 + 20 = 30.

Answer: complete cases give 33.3; IPW recovers the true 30.

Exercise 13 · Horvitz–Thompson versus Hájek in a household survey

Four sampled households have monthly food spending $Y = (10, 20, 30, \cdot)$ (in $10), where the fourth did not respond, so $R = (1, 1, 1, 0)$ . Their estimated response probabilities are $\pi = (0.5, 0.5, 0.8, 0.8)$ . Compute the Horvitz–Thompson and Hájek IPW estimates of mean spending.

Show solution

Step 1. Weighted values $R_iY_i/\pi_i$ : $10/0.5 = 20$ , $20/0.5 = 40$ , $30/0.8 = 37.5$ , and 0. Sum: 97.5.

Step 2. Horvitz–Thompson divides by $n = 4$ : $97.5/4 \approx 24.4$ .

Step 3. Hájek divides by the sum of the weights, $1/0.5 + 1/0.5 + 1/0.8 = 5.25$ : $97.5/5.25 \approx 18.6$ .

Answer: 24.4 versus 18.6 ($244 versus $186). In small samples they can differ a lot; the Hájek estimate always stays within the range of the observed values (here 10–30), so it is usually preferred.

Exercise 14 · Missing outcomes in regression

In a regression of wages on education, compare: (a) missingness that depends on education only; (b) missingness that depends on wages.

Show solution

(a) Missingness depends on education only

Step 1. Among people with the same education, respondents are a random subset, so $E[Y \mid X, R = 1] = E[Y \mid X]$ .

Step 2. The regression line is unchanged; you lose precision and perhaps part of the education range.

(b) Missingness depends on wages

Step 3. At each education level, high earners drop out. Respondents' average wage is pulled down most where wages are highest (high education).

Step 4. That flattens the line: attenuation bias.

Answer: selection on $X$ leaves the slope unbiased; selection on $Y$ biases it toward zero (see the MNAR setting in the Lab).

Exercise 15 · Potential outcomes as missing data

In an RCT with 50% treated, which potential outcome is missing for each unit, what is the "response probability" for $Y(1)$ , and which estimator of $E[Y(1)]$ does IPW give?

Show solution

Step 1. Treated units are missing $Y(0)$ ; controls are missing $Y(1)$ .

Step 2. $Y(1)$ is observed when $T = 1$ , which happens with probability 0.5 for everyone: the missingness is MCAR.

Step 3. IPW for $E[Y(1)]$ : $\frac1n\sum_iT_iY_i/0.5$ . The Hájek version divides by the realised number treated instead, giving the treated-group mean.

Step 4. If the propensity depends on covariates, $e(X) = P(T = 1 \mid X)$ , the weights become $T/e(X)$ for $Y(1)$ and $(1 - T)/(1 - e(X))$ for $Y(0)$ .

Answer: every causal estimate fills in missing potential outcomes; with a known or estimated propensity, Step 4 is the IPW estimator of the ATE (Day 44).

Verify in Python

import numpy as np

rng = np.random.default_rng(14)
n = 1_000_000
old = rng.random(n) < 0.5
Y = np.where(old, rng.normal(40, 5, n), rng.normal(20, 5, n))
pi = np.where(old, 0.8, 0.4)
R = rng.random(n) < pi                          # MAR: depends on age only
print("true mean        :", Y.mean().round(2))
print("complete cases   :", Y[R].mean().round(2))
print("IPW (HT)         :", (R * Y / pi).mean().round(2))
print("IPW (Hajek)      :", ((R * Y / pi).sum() / (R / pi).sum()).round(2))

# Rubin's rules (Exercise 10)
Q = np.array([10.2, 9.8, 10.5, 10.0, 9.5]); W = 0.25
B = Q.var(ddof=1); T = W + (1 + 1 / len(Q)) * B
print("pooled", Q.mean(), " total var", round(T, 3), " SE", round(np.sqrt(T), 3))

# MNAR attenuates a regression slope
educ = rng.normal(13, 2, 200_000)
wage = 5 + 2 * educ + rng.normal(0, 6, 200_000)
keep = rng.random(200_000) < 1 / (1 + np.exp((wage - 30) / 4))   # high earners drop out
print("slope full:", np.polyfit(educ, wage, 1)[0].round(3), " slope MNAR sample:", np.polyfit(educ[keep], wage[keep], 1)[0].round(3))

Key takeaways

MCAR is harmless for bias; MAR is fixable with the right covariates; MNAR needs extra assumptions or bounds.
IPW weights each respondent by the inverse of its response probability; it requires MAR and overlap.
Every causal estimate fills in missing potential outcomes, so these ideas reappear throughout the course.

Day 14. Missing Data & Inverse Probability Weighting

Theory

The response indicator

Inverse probability weighting (IPW)

Multiple imputation and Rubin's rules

Causal inference is a missing-data problem

Part 1 · Warm-up

Exercise 1 · Spot the mechanism (1)

Exercise 2 · Spot the mechanism (2)

Exercise 3 · Spot the mechanism (3)

Exercise 4 · An IPW weight

Exercise 5 · Complete cases under MCAR

Part 2 · Proofs

Exercise 6 · Complete-case mean under MCAR

Exercise 7 · IPW is unbiased under MAR

Exercise 8 · The bias under MNAR

Exercise 9 · Mean imputation understates variance

Exercise 10 · Rubin's rules

Part 3 · Applied

Exercise 11 · Bias from MNAR

Exercise 12 · Repairing MAR with weights

Exercise 13 · Horvitz–Thompson versus Hájek in a household survey

Exercise 14 · Missing outcomes in regression

Exercise 15 · Potential outcomes as missing data

Verify in Python

Key takeaways

Simulation Lab: Missing-Data Mechanisms