Day 42: Unconfoundedness & Overlap · Math for Causal Inference

Theory

From experiments to observational data

Without randomisation, treatment depends on choices. Selection on observables assumes that, once we compare units with the same covariates $X$ , assignment is as good as random.

Unconfoundedness (conditional independence, ignorability):

\big(Y(0), Y(1)\big) \perp T \mid X.

Overlap (positivity, common support): with the propensity score $e(x) = P(T = 1 \mid X = x)$ ,

0 < e(x) < 1 \quad\text{for all } x.

Together they identify the ATE through the adjustment formula (Day 6):

E[Y(1)] = E\big[E[Y \mid T = 1, X]\big], \qquad E[Y(0)] = E\big[E[Y \mid T = 0, X]\big].

For the ATT, it is enough that $Y(0) \perp T \mid X$ and $e(x) < 1$ : we only need to impute the treated's untreated outcomes.

Unconfoundedness is untestable…

It concerns $Y(0)$ and $Y(1)$ , which are never jointly observed. Its plausibility rests on knowing how treatment was assigned: which variables officials, doctors or applicants used. It is most credible when $X$ includes everything the decision-maker saw, especially lagged outcomes (past earnings, past test scores).

…but can be probed

Placebo outcomes: estimate the "effect" on an outcome measured before treatment (e.g. earnings two years earlier, not used as a control). A non-zero estimate signals hidden confounding.
Multiple control groups: results should not depend on which comparison group is used.
Sensitivity analysis (Day 95): how strong would an unobserved confounder need to be to explain the result away?

Overlap diagnostics

Normalised differences: $\Delta = (\bar x_1 - \bar x_0)/\sqrt{(s_1^2 + s_0^2)/2}$ . Values above about 0.25 signal that simple regression adjustment will rely on extrapolation (Imbens & Rubin 2015).
Propensity-score histograms by treatment group: regions with almost only treated or only control units are where comparisons are impossible.
Trimming: dropping units with $\hat e(x) \notin [0.1, 0.9]$ (Crump et al. 2009) improves precision but changes the estimand to the effect for the overlap population.

Part 1 · Warm-up

Exercise 1 · A normalised difference

Trainees earned $10k on average before the programme (sd 6k); a comparison group from a household survey earned $15k (sd 8k). Compute the normalised difference.

Show solution

Step 1. Pooled sd: $\sqrt{(36 + 64)/2} = \sqrt{50} \approx 7.07$ .

Step 2. $\Delta = (10 - 15)/7.07 \approx -0.71$ .

Answer: −0.71, far above the 0.25 rule of thumb: the groups are very different.

Exercise 2 · Extreme weights

A control unit has $e(x) = 0.98$ . If it is used to impute the treated's untreated outcomes with weight $e/(1 - e)$ (Day 44), what weight does it get?

Show solution

Step 1. $0.98/0.02 = 49$ .

Answer: this one control stands in for 49 treated units; estimates become fragile. Poor overlap shows up as extreme weights.

Exercise 3 · The adjustment formula

Smokers and non-smokers are offered a cessation app. Among young adults (60% of the population), app users quit at 30% and non-users at 20%; among older adults, 25% and 10%. Assuming unconfoundedness given age, compute the ATE.

Show solution

Step 1. Effect by age: young $0.30 - 0.20 = 0.10$ ; older $0.25 - 0.10 = 0.15$ .

Step 2. ATE $= 0.6(0.10) + 0.4(0.15) = 0.06 + 0.06 = 0.12$ .

Answer: 12 percentage points.

Exercise 4 · Trimming

10% of units have $\hat e < 0.1$ and 5% have $\hat e > 0.9$ . After trimming, what share of the sample remains, and for whom is the effect estimated?

Show solution

Step 1. Remaining: $1 - 0.10 - 0.05 = 85\%$ .

Answer: 85%; the estimate is the average effect for units whose covariates make both treatment and control plausible, not the ATE for everyone.

Exercise 5 · What to condition on

To study the effect of a microfinance loan on household consumption, which variables belong in $X$ : (a) baseline income, (b) household size at baseline, (c) business profits after the loan, (d) whether a relative is a loan officer?

Show solution

Step 1. (a), (b): pre-treatment and likely related to both borrowing and consumption: include.

Step 2. (d): pre-treatment and affects access to loans (and maybe consumption via networks): include.

Step 3. (c): post-treatment and affected by the loan: exclude (bad control).

Answer: (a), (b), (d).

Part 2 · Proofs

Exercise 6 · Identification of the ATE

Prove that unconfoundedness and overlap imply $E[Y(1)] = E[E[Y \mid T = 1, X]]$ , and similarly for $Y(0)$ .

Show solution

Step 1. LIE: $E[Y(1)] = E\big[E[Y(1) \mid X]\big]$ .

Step 2. Unconfoundedness: $E[Y(1) \mid X] = E[Y(1) \mid T = 1, X]$ .

Step 3. Consistency: among the treated $Y = Y(1)$ , so $E[Y(1) \mid T = 1, X] = E[Y \mid T = 1, X]$ .

Step 4. Overlap ( $e(X) > 0$ ) guarantees treated units exist at every $X$ , so this conditional mean is defined.

Step 5. Combine: $E[Y(1)] = E[E[Y \mid T = 1, X]]$ . The $Y(0)$ case uses $e(X) < 1$ . $\blacksquare$

Exercise 7 · Weaker conditions for the ATT

Prove that $Y(0) \perp T \mid X$ and $e(X) < 1$ suffice to identify the ATT.

Show solution

Step 1. $\text{ATT} = E[Y \mid T = 1] - E[Y(0) \mid T = 1]$ ; the first term is observed.

Step 2. LIE over the treated's covariates: $E[Y(0) \mid T = 1] = E\big[E[Y(0) \mid X, T = 1] \mid T = 1\big]$ .

Step 3. $Y(0) \perp T \mid X$ gives $E[Y(0) \mid X, T = 1] = E[Y(0) \mid X, T = 0] = E[Y \mid X, T = 0]$ .

Step 4. $e(X) < 1$ guarantees control units exist at every $X$ found among the treated.

Answer: $\text{ATT} = E[Y \mid T = 1] - E\big[E[Y \mid X, T = 0] \mid T = 1\big]$ . Nothing is assumed about $Y(1)$ or about $e(X) > 0$ . $\blacksquare$

Exercise 8 · Overlap failure means extrapolation

Suppose $e(x^*) = 1$ for some covariate value $x^*$ with positive probability. Show that $E[Y(0) \mid X = x^*]$ is not identified without further assumptions.

Show solution

Step 1. At $x^*$ every unit is treated, so no $Y(0)$ is ever observed there.

Step 2. Two data-generating processes that agree everywhere except on $E[Y(0) \mid X = x^*]$ produce identical observable data.

Step 3. So the data cannot distinguish them: not identified. A regression model fills the gap only by extrapolating its functional form.

Answer: without overlap, estimates rest on modelling assumptions, not on comparisons.

Exercise 9 · Placebo outcomes

Let $Y^{pre}$ be an outcome measured before treatment, so the true effect on it is zero. Show that if unconfoundedness held for $Y^{pre}$ (given $X$ excluding $Y^{pre}$ ), the adjusted "effect" on $Y^{pre}$ would be zero, and explain the logic of the test.

Show solution

Step 1. Treatment cannot affect the past: $Y^{pre}(1) = Y^{pre}(0)$ .

Step 2. If $Y^{pre} \perp T \mid X$ , the adjustment formula gives $E[E[Y^{pre} \mid T = 1, X]] - E[E[Y^{pre} \mid T = 0, X]] = 0$ .

Step 3. A non-zero estimate shows that, given $X$ , treated and controls still differ systematically.

Answer: the placebo checks for confounding of the kind that also affects earlier outcomes; passing it is reassuring, not proof.

Exercise 10 · Bad controls break unconfoundedness

Training $T$ is randomised, and it raises the chance of taking a full-time job $F$ . Show that comparing wages of trained and untrained workers within $F = 1$ can be biased even though $T$ was random.

Show solution

Step 1. $F$ has potential values $F(1), F(0)$ . Trained full-timers have $F(1) = 1$ ; untrained full-timers have $F(0) = 1$ .

Step 2. Write $E[Y \mid T = 1, F = 1] - E[Y \mid T = 0, F = 1]$ . With randomised $T$ this equals $E[Y(1) \mid F(1) = 1] - E[Y(0) \mid F(0) = 1]$ .

Step 3. Workers who get full-time jobs only with training are likely less skilled than those who would get them anyway, so the two groups differ in $Y(0)$ .

Answer: conditioning on a post-treatment variable compares different populations; it breaks the independence that randomisation created.

Part 3 · Applied

Exercise 11 · Survey comparison groups for a training programme

An evaluation compares 185 training participants with 2,490 household-survey respondents. Normalised differences: age −0.9, education −0.7, baseline earnings −1.6, married −1.2. What do these imply, and what should the analyst do before estimating?

Show solution

Step 1. All differences far exceed 0.25: the survey respondents are much older, better educated, richer and more often married.

Step 2. Many survey respondents have covariate values with almost no participants (propensity near 0): weak overlap.

Step 3. A linear regression would extrapolate across these gaps.

Answer: estimate propensity scores, inspect overlap, trim or match to comparable respondents, and check results with placebo outcomes. This is the lesson of LaLonde (1986) and Dehejia & Wahba (1999).

Exercise 12 · Hospital quality

A ministry ranks hospitals by 30-day mortality, adjusting for age, sex and diagnosis. Why might unconfoundedness fail, and what data would help?

Show solution

Step 1. "Treatment" is the hospital attended; sicker patients may go to specialist hospitals because of severity not captured by diagnosis codes.

Step 2. Unobserved severity raises mortality and affects which hospital is chosen: unconfoundedness given the coded $X$ fails, penalising good specialist hospitals.

Step 3. Richer clinical data (lab values, vital signs on arrival), or quasi-random assignment such as ambulance rotation or distance to hospitals, would help.

Answer: adjusted rankings are only as good as the severity measures.

Exercise 13 · What trimming estimates

In a study of a cash-transfer programme's effect on child schooling, households with $\hat e > 0.9$ are the poorest (almost all treated). After trimming them, what estimand do you get, and how should it be reported?

Show solution

Step 1. The poorest households are removed, so the estimate is the average effect among moderately poor households where both treated and untreated exist.

Step 2. If effects are largest for the poorest, the trimmed estimate understates the ATE.

Answer: report it as the effect for the overlap population, describe who was trimmed, and avoid presenting it as the programme-wide ATE.

Exercise 14 · How strong would a confounder need to be?

An observational study finds that preschool attendance raises later earnings by 8%. Critics point to unmeasured parental motivation. Describe the logic of a sensitivity analysis.

Show solution

Step 1. Posit a confounder $U$ and parametrise its strength: how much it raises the odds of attending, and how much it raises earnings.

Step 2. For each pair of strengths, compute the bias (Day 34) and the corrected estimate.

Step 3. Report the smallest confounder that would make the effect zero, and compare it with the strength of observed covariates (e.g. "as strong as parental education").

Answer: sensitivity analysis replaces "untestable" with "how wrong would we have to be" (Day 95).

Exercise 15 · When treatment follows a rule

A subsidy is given to every household whose poverty score is below 30. Is unconfoundedness given the score plausible? Is overlap?

Show solution

Step 1. Treatment is a deterministic function of the score, so given the score there is no other selection: unconfoundedness holds by design.

Step 2. But $e(\text{score}) = 1$ below 30 and 0 above: overlap fails everywhere.

Step 3. Comparisons are possible only near the cut-off, by assuming outcomes would be continuous in the score there.

Answer: unconfoundedness yes, overlap no: this is the regression discontinuity design (Days 73–77).

Verify in Python

import numpy as np

rng = np.random.default_rng(42)
n = 20_000
x = rng.normal(0, 1, n)                                   # e.g. standardised baseline earnings
for strength in (0.5, 1.5, 3.0):
    e = 1 / (1 + np.exp(-strength * x))
    T = rng.random(n) < e
    nd = (x[T].mean() - x[~T].mean()) / np.sqrt((x[T].var() + x[~T].var()) / 2)
    out = np.mean((e < 0.1) | (e > 0.9))
    w = e[~T] / (1 - e[~T])
    print(f"selection strength {strength}: normalised diff {nd:.2f}, share outside [0.1, 0.9] {out:.2f}, "
          f"max ATT weight on a control {w.max():.0f}")

# Adjustment formula (Exercise 3)
print("ATE by age adjustment:", round(0.6 * (0.30 - 0.20) + 0.4 * (0.25 - 0.10), 3))

Key takeaways

Unconfoundedness plus overlap identify the ATE; the ATT needs only the conditions on Y(0) and e(x) < 1.
Unconfoundedness is untestable but can be probed with placebo outcomes and multiple comparison groups.
Check overlap first; trimming improves precision but changes the estimand.

Day 42. Unconfoundedness & Overlap

Theory

From experiments to observational data

Unconfoundedness is untestable…

…but can be probed

Overlap diagnostics

Part 1 · Warm-up

Exercise 1 · A normalised difference

Exercise 2 · Extreme weights

Exercise 3 · The adjustment formula

Exercise 4 · Trimming

Exercise 5 · What to condition on

Part 2 · Proofs

Exercise 6 · Identification of the ATE

Exercise 7 · Weaker conditions for the ATT

Exercise 8 · Overlap failure means extrapolation

Exercise 9 · Placebo outcomes

Exercise 10 · Bad controls break unconfoundedness

Part 3 · Applied

Exercise 11 · Survey comparison groups for a training programme

Exercise 12 · Hospital quality

Exercise 13 · What trimming estimates

Exercise 14 · How strong would a confounder need to be?

Exercise 15 · When treatment follows a rule

Verify in Python

Key takeaways

Simulation Lab: Overlap & Trimming