Day 43: The Propensity Score Theorem · Math for Causal Inference

Theory

The problem of many covariates

With many covariates, exact matching on $X$ is impossible: almost every unit has a unique covariate combination. Rosenbaum & Rubin (1983) showed that a single summary suffices.

Two results

The propensity score is $e(X) = P(T = 1 \mid X)$ .

Balancing property: $X \perp T \mid e(X)$ . Among units with the same propensity score, the distribution of covariates is the same for treated and controls.
Propensity score theorem: if $(Y(0), Y(1)) \perp T \mid X$ , then $(Y(0), Y(1)) \perp T \mid e(X)$ .

So instead of adjusting for a vector $X$ , it is enough to adjust for the scalar $e(X)$ : match, stratify or weight on it.

Using it in practice

Estimate $e(X)$ , typically by logit (Day 11) with flexible terms (interactions, squares) or machine learning.
Check overlap (Day 42) and trim if needed.
Check balance: within propensity strata (or after weighting), normalised differences of each covariate should be small. Balance is the diagnostic for whether the score model is good enough, not its fit or its $R^2$ .
Estimate effects: subclassification (5 strata typically remove about 90% of the bias from a single covariate, Cochran 1968), matching (Day 81), or weighting (Day 44).

What it cannot do

The propensity score only balances the covariates used to build it. It is a tool for implementing unconfoundedness, not a substitute for it.

Part 1 · Warm-up

Exercise 1 · A propensity score from a logit

A logit for enrolling in a training programme has $\log\frac{e}{1 - e} = 1.0 - 0.1 \times \text{age} + 0.5 \times \text{unemployed}$ . Compute $e$ for a 30-year-old unemployed worker.

Show solution

Step 1. Index: $1.0 - 3.0 + 0.5 = -1.5$ .

Step 2. $e = 1/(1 + e^{1.5}) = 1/(1 + 4.48) \approx 0.18$ .

Answer: an 18% chance of enrolling.

Exercise 2 · Same score, different covariates

A 25-year-old employed worker has index $1.0 - 2.5 + 0 = -1.5$ . Compare with Exercise 1. What does the balancing property say about such pairs?

Show solution

Step 1. Both workers have the same index, hence $e \approx 0.18$ .

Step 2. They differ in age and employment, but are equally likely to enrol.

Answer: among units with $e = 0.18$ , the mix of (age, employment) profiles is the same for enrollees and non-enrollees, so comparing them does not confound the effect with these covariates.

Exercise 3 · Subclassification estimate

Five propensity strata of equal size have within-stratum treated-minus-control differences in earnings ($1,000) of 0.5, 1.0, 1.5, 1.2, 2.0. Estimate the ATE.

Show solution

Step 1. Equal stratum sizes give equal weights of 1/5.

Step 2. ATE $\approx (0.5 + 1.0 + 1.5 + 1.2 + 2.0)/5 = 6.2/5 = 1.24$ .

Answer: about $1,240.

Exercise 4 · ATT by subclassification

In Exercise 3, the strata contain 5%, 10%, 20%, 25% and 40% of all treated units. Estimate the ATT.

Show solution

Step 1. Weight strata by their share of treated units.

Step 2. ATT $\approx 0.05(0.5) + 0.10(1.0) + 0.20(1.5) + 0.25(1.2) + 0.40(2.0)$ .

Step 3. $= 0.025 + 0.10 + 0.30 + 0.30 + 0.80 = 1.525$ .

Answer: about $1,525: the treated sit mostly in high-propensity strata, where effects are larger.

Exercise 5 · Balance within a stratum

In one propensity stratum, treated workers average 11.2 years of schooling (sd 2.5) and controls 10.9 (sd 2.4). Compute the normalised difference. Is balance acceptable?

Show solution

Step 1. Pooled sd: $\sqrt{(6.25 + 5.76)/2} \approx 2.45$ .

Step 2. $\Delta = 0.3/2.45 \approx 0.12$ .

Answer: 0.12 < 0.25: acceptable. Check every covariate in every stratum.

Part 2 · Proofs

Exercise 6 · The balancing property

Prove $P(T = 1 \mid X, e(X)) = P(T = 1 \mid e(X))$ , i.e. $X \perp T \mid e(X)$ .

Show solution

Step 1. $e(X)$ is a function of $X$ , so conditioning on $(X, e(X))$ is conditioning on $X$ : $P(T = 1 \mid X, e(X)) = e(X)$ .

Step 2. LIE: $P(T = 1 \mid e(X)) = E[T \mid e(X)] = E\big[E[T \mid X] \mid e(X)\big] = E[e(X) \mid e(X)] = e(X)$ .

Step 3. Both equal $e(X)$ , so knowing $X$ beyond $e(X)$ does not change the treatment probability: $T \perp X \mid e(X)$ . $\blacksquare$

Exercise 7 · The propensity score theorem

Assume $(Y(0), Y(1)) \perp T \mid X$ . Prove $P(T = 1 \mid Y(0), Y(1), e(X)) = e(X)$ , i.e. unconfoundedness given $e(X)$ .

Show solution

Write $\mathbf Y = (Y(0), Y(1))$ .

Step 1. LIE: $P(T = 1 \mid \mathbf Y, e(X)) = E\big[E[T \mid \mathbf Y, X] \mid \mathbf Y, e(X)\big]$ (conditioning on $X$ refines conditioning on $e(X)$ ).

Step 2. Unconfoundedness given $X$ : $E[T \mid \mathbf Y, X] = E[T \mid X] = e(X)$ .

Step 3. So $P(T = 1 \mid \mathbf Y, e(X)) = E[e(X) \mid \mathbf Y, e(X)] = e(X)$ .

Step 4. This does not depend on $\mathbf Y$ , so $T \perp \mathbf Y \mid e(X)$ . $\blacksquare$

Exercise 8 · Adjusting for the score identifies the ATE

Using Exercise 7, show $\text{ATE} = E\big[E[Y \mid T = 1, e(X)] - E[Y \mid T = 0, e(X)]\big]$ .

Show solution

Step 1. Apply Day 42, Exercise 6 with $e(X)$ playing the role of $X$ : unconfoundedness holds given $e(X)$ (Exercise 7), and overlap $0 < e < 1$ holds by assumption.

Step 2. So $E[Y(t)] = E\big[E[Y \mid T = t, e(X)]\big]$ for $t = 0, 1$ ; subtract. $\blacksquare$

Exercise 9 · Any balancing score works

A balancing score is any $b(X)$ with $X \perp T \mid b(X)$ . Show that $X$ itself is a balancing score, and that $e(X)$ is the coarsest one (it is a function of every balancing score).

Show solution

Step 1. Given $X$ , $X$ is fixed, so trivially independent of $T$ : $X$ is a balancing score (the finest).

Step 2. If $b(X)$ is balancing, then $e(X) = P(T = 1 \mid X) = P(T = 1 \mid X, b(X)) = P(T = 1 \mid b(X))$ , using the balancing property in the last step.

Step 3. So $e(X)$ can be computed from $b(X)$ : it is a function of every balancing score. $\blacksquare$

The propensity score achieves the maximum dimension reduction.

Exercise 10 · The propensity score does not fix hidden bias

Suppose $T$ depends on $X$ and an unobserved $U$ that also affects $Y$ . Explain why adjusting for $e(X) = P(T = 1 \mid X)$ does not remove confounding by $U$ .

Show solution

Step 1. Exercise 7 needed unconfoundedness given $X$ ; here it fails because of $U$ .

Step 2. Within a stratum of $e(X)$ , treated units still have different $U$ (hence different $Y(0)$ ) than controls.

Answer: the propensity score balances only what goes into it. Good balance on $X$ says nothing about $U$ .

Part 3 · Applied

Exercise 11 · Subclassification in a training study

A training study estimates propensity scores and forms five strata. Within strata the effects and sizes are:

Stratum	Treated	Controls	Effect ($1,000)
1	5	395	0.4
2	10	190	0.8
3	20	80	1.2
4	40	40	1.6
5	60	15	2.0

Estimate the ATE and the ATT.

Show solution

Step 1. Stratum sizes: 400, 200, 100, 80, 75; total 855. Treated: 135.

Step 2. ATE weights $N_s/855$ : $\frac{400(0.4) + 200(0.8) + 100(1.2) + 80(1.6) + 75(2.0)}{855} = \frac{160 + 160 + 120 + 128 + 150}{855} = \frac{718}{855} \approx 0.84$ .

Step 3. ATT weights $N_{1s}/135$ : $\frac{5(0.4) + 10(0.8) + 20(1.2) + 40(1.6) + 60(2.0)}{135} = \frac{2 + 8 + 24 + 64 + 120}{135} = \frac{218}{135} \approx 1.61$ .

Answer: ATE ≈ $840, ATT ≈ $1,610: the treated are concentrated where effects are large. Stratum 1's 5 treated and stratum 5's 15 controls signal thin overlap at the extremes.

Exercise 12 · Checking the score model

After subclassification, baseline earnings are balanced in four strata but in stratum 5 treated workers earn $3,000 more (normalised difference 0.6). What should the analyst do?

Show solution

Step 1. Imbalance means the score model does not capture how earnings relate to treatment at high propensities.

Step 2. Re-specify: add earnings squared, interactions (earnings × unemployed), or split stratum 5.

Step 3. Re-check balance; iterate. Do not look at outcomes while doing this, so the model is chosen for balance, not for the answer.

Answer: refine the propensity model until balance is achieved, blind to outcomes.

Exercise 13 · Smoking and birth weight

A study of maternal smoking and birth weight uses a propensity score based on age, education, parity and prenatal visits. Name a plausible unobserved confounder and the likely direction of bias.

Show solution

Step 1. Candidates: alcohol use, diet, stress, or partner's smoking: correlated with smoking and harmful to the baby.

Step 2. These make smokers' babies lighter even apart from smoking, so the estimated harm of smoking is overstated (biased away from zero).

Answer: e.g. alcohol use; the propensity score cannot fix it, but a sensitivity analysis can show how strong it would need to be.

Exercise 14 · Many covariates, few treated

A hospital study has 80 treated patients and 60 candidate covariates. Why can a saturated logit for the propensity score be a problem, and what are the alternatives?

Show solution

Step 1. With 60 covariates and 80 treated, the logit overfits: estimated scores pile up near 0 and 1, sometimes with perfect separation (Day 25).

Step 2. Overfitted scores exaggerate apparent non-overlap and produce extreme weights.

Step 3. Alternatives: choose covariates by subject-matter knowledge (those affecting both treatment and outcome), penalised logit (ridge or lasso), or methods that target balance directly (entropy balancing, Day 84).

Answer: a propensity model is a means to balance, not a prediction contest.

Exercise 15 · Is a high-AUC propensity model good news?

A propensity model predicts enrolment in a job-search programme almost perfectly (area under the ROC curve 0.98). Is that good or bad for estimating the programme's effect?

Show solution

Step 1. Near-perfect prediction means scores close to 0 or 1 for most units.

Step 2. That signals poor overlap: few comparable treated and control units.

Answer: bad. Unlike in prediction tasks, a highly accurate propensity model warns that causal comparisons will rely on few units or on extrapolation.

Verify in Python

import numpy as np
from scipy import optimize

rng = np.random.default_rng(43)
n = 20_000
age = rng.normal(35, 10, n); unemp = rng.random(n) < 0.4
X = np.column_stack([np.ones(n), age, unemp])
T = rng.random(n) < 1 / (1 + np.exp(-(1.0 - 0.06 * age + 1.0 * unemp)))
y0 = 20 + 0.3 * age - 5 * unemp + rng.normal(0, 4, n)
y1 = y0 + 1 + 2 * unemp                                       # effect larger for the unemployed
y = np.where(T, y1, y0)

nll = lambda b: -np.sum(T * (X @ b) - np.log1p(np.exp(X @ b)))
b = optimize.minimize(nll, np.zeros(3)).x
e = 1 / (1 + np.exp(-X @ b))
strata = np.digitize(e, np.quantile(e, [0.2, 0.4, 0.6, 0.8]))
ate = sum((strata == s).mean() * (y[T & (strata == s)].mean() - y[~T & (strata == s)].mean()) for s in range(5))
att = sum((T & (strata == s)).sum() / T.sum() * (y[T & (strata == s)].mean() - y[~T & (strata == s)].mean()) for s in range(5))
# five strata remove most, but not all, of the confounding bias
print(f"naive {y[T].mean() - y[~T].mean():.2f}; subclassification ATE {ate:.2f} (truth {(y1 - y0).mean():.2f}), "
      f"ATT {att:.2f} (truth {(y1 - y0)[T].mean():.2f})")
for name, v in (("age", age), ("unemployed", unemp.astype(float))):
    nd = [(v[T & (strata == s)].mean() - v[~T & (strata == s)].mean()) / v.std() for s in range(5)]
    print(f"normalised differences within strata, {name}:", np.round(nd, 2))

print("Exercise 11: ATE", round(718 / 855, 3), " ATT", round(218 / 135, 3))

Key takeaways

Conditional on the propensity score, covariates are balanced between treated and controls.
If treatment is unconfounded given X, it is unconfounded given e(X): one number replaces many covariates.
Judge a propensity model by the balance it achieves, never by its predictive accuracy; it cannot fix unobserved confounding.

Day 43. The Propensity Score Theorem

Theory

The problem of many covariates

Two results

Using it in practice

What it cannot do

Part 1 · Warm-up

Exercise 1 · A propensity score from a logit

Exercise 2 · Same score, different covariates

Exercise 3 · Subclassification estimate

Exercise 4 · ATT by subclassification

Exercise 5 · Balance within a stratum

Part 2 · Proofs

Exercise 6 · The balancing property

Exercise 7 · The propensity score theorem

Exercise 8 · Adjusting for the score identifies the ATE

Exercise 9 · Any balancing score works

Exercise 10 · The propensity score does not fix hidden bias

Part 3 · Applied

Exercise 11 · Subclassification in a training study

Exercise 12 · Checking the score model

Exercise 13 · Smoking and birth weight

Exercise 14 · Many covariates, few treated

Exercise 15 · Is a high-AUC propensity model good news?

Verify in Python

Key takeaways

Simulation Lab: Balancing on the Propensity Score