Day 53: Adjustment Sets & Bad Controls · Math for Causal Inference

Theory

A catalogue of controls

Let $X$ be the treatment and $Y$ the outcome. For a candidate control $Z$ (and possibly unobserved $U$ ):

Structure	Verdict	Why
$X \leftarrow Z \to Y$	good	blocks a back-door path
$X \leftarrow Z \leftarrow U \to Y$	good	blocks the path through $U$ (a proxy)
$Z \to Y$ only	neutral, helps precision	explains outcome variance
$Z \to X$ only	neutral, hurts precision	removes treatment variation
$Z \to X$ , $X \leftarrow U \to Y$	bad: bias amplification	removes clean variation, keeps confounded variation
$X \to Z \to Y$	bad (for the total effect)	blocks part of the effect
$X \to Z \leftarrow Y$ , or $X \to Z \leftarrow U \to Y$	bad	conditioning on a collider
$X \leftarrow U_1 \to Z \leftarrow U_2 \to Y$	bad: M-bias	pre-treatment collider
$Y \to Z$	bad	selection on the outcome

"Pre-treatment" is not the same as "safe": the M-structure collider can be measured before treatment.

Precision in a linear model

In $Y = \tau X + \boldsymbol\gamma^\top\mathbf Z + \varepsilon$ estimated by OLS, Frisch–Waugh–Lovell gives

\operatorname{Var}(\hat\tau) \approx \frac{\sigma^2_{Y \mid X, \mathbf Z}}{n\,\operatorname{Var}(X \mid \mathbf Z)}.

An outcome predictor shrinks the numerator: more precision.
A treatment predictor shrinks the denominator: less precision (the variance inflation factor $1/(1 - R^2_{X \mid \mathbf Z})$ ).

Choosing among admissible sets

Several sets may satisfy the back-door criterion. In linear models the asymptotically most precise one is the optimal adjustment set: roughly, the parents of $Y$ (and of mediators on the causal path) that are not themselves descendants of $X$ , excluding $X$ . Practical rule: prefer confounders close to the outcome over confounders close to the treatment.

The Table 2 fallacy

A regression designed to estimate the effect of $X$ has coefficients on the controls too. They are generally not causal effects of the controls: $X$ may be a mediator or collider for them. Report only the coefficient the design was built for.

Part 1 · Warm-up

Exercise 1 · Classify the controls

For the effect of $X$ on $Y$ , classify $Z$ : (a) $Z \to X$ , $Z \to Y$ ; (b) $X \to Z$ , $Y \to Z$ ; (c) $Z \to Y$ only; (d) $Z \to X$ only, no confounding; (e) $X \to Z \to Y$ .

Show solution

Step 1. (a) Confounder: good.

Step 2. (b) Collider: bad.

Step 3. (c) Outcome predictor: neutral, improves precision.

Step 4. (d) Instrument-like: neutral for bias, reduces precision.

Step 5. (e) Mediator: bad for the total effect.

Exercise 2 · Precision from an outcome predictor

$Y = \tau X + 2Z + \varepsilon$ , with $Z \perp\!\!\!\perp X$ , $\operatorname{Var}(Z) = 1$ , $\operatorname{Var}(\varepsilon) = 1$ . By what factor does including $Z$ reduce the standard error of $\hat\tau$ ?

Show solution

Step 1. Without $Z$ : residual variance $4\operatorname{Var}(Z) + 1 = 5$ .

Step 2. With $Z$ : residual variance 1.

Step 3. $\operatorname{Var}(X \mid Z) = \operatorname{Var}(X)$ since they are independent, so the denominator is unchanged.

Answer: variance falls by a factor of 5; the SE by $\sqrt5 \approx 2.24$ .

Exercise 3 · Precision lost to an instrument

$X = Z + u$ with $\operatorname{Var}(Z) = \operatorname{Var}(u) = 1$ , and $Z$ affects $Y$ only through $X$ (no confounding). By what factor does controlling for $Z$ raise $\operatorname{Var}(\hat\tau)$ ?

Show solution

Step 1. $\operatorname{Var}(X) = 2$ and $\operatorname{Var}(X \mid Z) = \operatorname{Var}(u) = 1$ .

Step 2. The residual variance of $Y$ is unchanged (Z has no direct effect on $Y$ ).

Answer: the variance doubles ( $R^2_{X \mid Z} = 0.5$ ); the SE rises by $\sqrt2$ .

Exercise 4 · M-bias

In $U_1 \to X$ , $U_1 \to M$ , $U_2 \to M$ , $U_2 \to Y$ , $X \to Y$ , with $U_1, U_2$ unobserved, is $\varnothing$ admissible? Is $\{M\}$ ?

Show solution

Step 1. The only back-door path is $X \leftarrow U_1 \to M \leftarrow U_2 \to Y$ , with a collider at $M$ .

Step 2. Given $\varnothing$ it is blocked: $\varnothing$ is admissible.

Step 3. Given $M$ it opens: $\{M\}$ is not admissible, although $M$ is pre-treatment and correlated with both $X$ and $Y$ .

Exercise 5 · Controlling for a consequence of the outcome

Why is it a mistake to control for $D$ when $Y \to D$ (for example, $D$ = hospital discharge diagnosis, $Y$ = disease severity)?

Show solution

Step 1. $X \to Y \to D$ : $D$ is a descendant of $X$ , violating the back-door condition.

Step 2. Conditioning on $D$ partially conditions on $Y$ itself: among units with the same $D$ , variation in $Y$ is restricted, which attenuates (or distorts) the estimated effect.

Answer: a descendant of the outcome is a bad control.

Part 2 · Proofs

Exercise 6 · The variance formula

In $Y = \tau X + \boldsymbol\gamma^\top\mathbf Z + \varepsilon$ with homoskedastic $\varepsilon$ , show $\operatorname{Var}(\hat\tau \mid X, \mathbf Z) = \sigma^2/\sum_i \tilde X_i^2$ , where $\tilde X$ is $X$ residualised on $\mathbf Z$ , and conclude the approximation in the theory.

Show solution

Step 1. FWL: $\hat\tau = \sum_i \tilde X_i Y_i/\sum_i \tilde X_i^2$ .

Step 2. Substitute $Y_i = \tau X_i + \boldsymbol\gamma^\top\mathbf Z_i + \varepsilon_i$ ; since $\tilde X$ is orthogonal to $\mathbf Z$ and $\sum \tilde X_i X_i = \sum \tilde X_i^2$ , $\hat\tau = \tau + \sum \tilde X_i\varepsilon_i/\sum \tilde X_i^2$ .

Step 3. $\operatorname{Var}(\hat\tau \mid X, \mathbf Z) = \sigma^2\sum \tilde X_i^2/(\sum \tilde X_i^2)^2 = \sigma^2/\sum \tilde X_i^2$ .

Step 4. $\sum \tilde X_i^2 \approx n\operatorname{Var}(X \mid \mathbf Z)$ for large $n$ . $\blacksquare$

Exercise 7 · Bias amplification

Let $X = aZ + bU + e_X$ and $Y = \tau X + cU + e_Y$ , with $Z, U, e_X, e_Y$ independent, $\operatorname{Var}(Z) = \operatorname{Var}(U) = 1$ , $\operatorname{Var}(e_X) = s^2$ , and $U$ unobserved. Show the bias of $\hat\tau$ is $cb/(a^2 + b^2 + s^2)$ without $Z$ and $cb/(b^2 + s^2)$ with $Z$ .

Show solution

Step 1. Without $Z$ : the slope is $\tau + c\operatorname{Cov}(X, U)/\operatorname{Var}(X) = \tau + cb/(a^2 + b^2 + s^2)$ (Day 50, Exercise 8).

Step 2. With $Z$ : by FWL, use $\tilde X = X - aZ = bU + e_X$ .

Step 3. Slope $= \operatorname{Cov}(\tilde X, Y)/\operatorname{Var}(\tilde X)$ , and $\operatorname{Cov}(\tilde X, Y) = \tau\operatorname{Var}(\tilde X) + cb$ .

Step 4. So the bias is $cb/(b^2 + s^2)$ , which is larger: controlling for $Z$ removed clean variation ( $a^2$ ) from the denominator. $\blacksquare$

Exercise 8 · Outcome predictors do not change the estimand

Assume $\mathbf Z$ is admissible and $W$ is independent of $(X, \mathbf Z)$ and affects only $Y$ . Prove that $\mathbf Z \cup \{W\}$ is admissible.

Show solution

Step 1. Condition 1: $W$ has no parents among the descendants of $X$ (it is independent of $X$ with arrows only into $Y$ ), so it is not a descendant of $X$ .

Step 2. Condition 2: every path through $W$ is $\cdots Y \leftarrow W$ with $W$ as an endpoint-adjacent node; $W$ has no other neighbours, so no back-door path from $X$ passes through $W$ .

Step 3. Conditioning on $W$ cannot open a path: $W$ is not a collider or a descendant of one on any $X$ – $Y$ path.

Step 4. Paths blocked by $\mathbf Z$ remain blocked. $\blacksquare$

Exercise 9 · Controlling for a confounded mediator

Let $M = aX + U + e_M$ and $Y = dX + bM + cU + e_Y$ , with $X, U, e_M, e_Y$ independent, $\operatorname{Var}(U) = v_U$ , $\operatorname{Var}(e_M) = v_M$ . Show that the coefficient on $X$ in the regression of $Y$ on $(X, M)$ is $d - cak$ , with $k = v_U/(v_U + v_M)$ .

Show solution

Step 1. Given $X$ and $M$ , we know $U + e_M = M - aX$ . By normal (or linear) projection, $E[U \mid X, M] = k(M - aX)$ .

Step 2. $E[Y \mid X, M] = dX + bM + cE[U \mid X, M] = dX + bM + ck(M - aX)$ .

Step 3. Collect terms: $(d - cak)X + (b + ck)M$ .

Answer: the coefficient is the direct effect $d$ only if $c = 0$ (no mediator–outcome confounding). Otherwise $M$ is a collider on $X \to M \leftarrow U \to Y$ . $\blacksquare$

Exercise 10 · The size of M-bias

In Exercise 4 with linear equations $X = \alpha U_1 + e_X$ , $M = U_1 + U_2 + e_M$ , $Y = \tau X + \beta U_2 + e_Y$ (all disturbances independent with variance 1), show that controlling for $M$ adds bias $-\alpha\beta/(2\alpha^2 + 3)$ .

Show solution

Step 1. $E[Y \mid X, M] = \tau X + \beta E[U_2 \mid X, M]$ .

Step 2. $\operatorname{Var}(X) = \alpha^2 + 1$ , $\operatorname{Var}(M) = 3$ , $\operatorname{Cov}(X, M) = \alpha$ ; $\operatorname{Cov}(U_2, X) = 0$ , $\operatorname{Cov}(U_2, M) = 1$ .

Step 3. Projection coefficients: $(0, 1)\begin{pmatrix} \alpha^2 + 1 & \alpha \\ \alpha & 3 \end{pmatrix}^{-1} = \dfrac{(-\alpha,\; \alpha^2 + 1)}{3(\alpha^2 + 1) - \alpha^2}$ .

Step 4. The coefficient on $X$ in $E[U_2 \mid X, M]$ is $-\alpha/(2\alpha^2 + 3)$ .

Answer: bias $-\alpha\beta/(2\alpha^2 + 3)$ ; with $\alpha = \beta = 1$ it is −0.2. Usually modest, but real. $\blacksquare$

Part 3 · Applied

Exercise 11 · Medicaid expansion and mortality

You study the effect of a state's Medicaid expansion $X$ on county mortality $Y$ . Classify each candidate control: (a) pre-expansion county poverty rate; (b) post-expansion insurance coverage; (c) post-expansion number of hospital closures, affected by both expansion and by local death rates through hospital finances; (d) county rainfall; (e) pre-expansion mortality trend.

Show solution

Step 1. (a) Affects expansion decisions and mortality: confounder, include.

Step 2. (b) Mediator: expansion works through coverage. Exclude for the total effect.

Step 3. (c) Descendant of $X$ and of $Y$ : collider-like and a descendant of the outcome. Exclude.

Step 4. (d) Unrelated to expansion; weakly predicts mortality at most. Harmless; include only if it explains outcome variance.

Step 5. (e) Strong predictor of mortality and plausibly of expansion: include (good for bias and precision).

Exercise 12 · A cash-transfer RCT

In a randomised cash-transfer trial with 2,000 households, baseline consumption explains 60% of the variance of endline consumption. A field officer also proposes controlling for the household's distance to the payment point, which affected take-up but not consumption. What happens to the standard error of the effect of being offered the transfer with each control?

Show solution

Step 1. Offer is randomised, so both controls leave the estimand unchanged.

Step 2. Baseline consumption: residual variance falls to 40%; SE falls by $\sqrt{0.4} \approx 0.63$ , i.e. 37% smaller.

Step 3. Distance: it predicts take-up, not the random offer itself, and not consumption; it neither helps nor hurts the offer coefficient (beyond a tiny degrees-of-freedom cost).

Answer: control for baseline consumption; distance is irrelevant for the intention-to-treat effect. (If the treatment were take-up rather than the offer, distance would be an instrument and should not be a control.)

Exercise 13 · Bias amplification in returns to schooling

Schooling $S = 0.8D + 1.0A + e_S$ , where $D$ is distance to the nearest college (standardised, negative effect coded positive), $A$ unobserved ability (variance 1), $\operatorname{Var}(e_S) = 1$ ; log wage $Y = 0.08S + 0.10A + e_Y$ . Compute the ability bias with and without controlling for $D$ .

Show solution

Step 1. Use Exercise 7 with $a = 0.8$ , $b = 1$ , $s^2 = 1$ , $c = 0.10$ .

Step 2. Without $D$ : $0.10 \times 1/(0.64 + 1 + 1) = 0.10/2.64 \approx 0.038$ .

Step 3. With $D$ : $0.10/(1 + 1) = 0.050$ .

Answer: controlling for distance raises the bias from 3.8 to 5.0 points (estimated return 11.8% to 13.0% instead of 8%). Distance belongs in an IV analysis (Phase 5), not in the control list.

Exercise 14 · Choosing the most precise admissible set

In a smoking-cessation study, $A$ (age) $\to X$ (counselling), $A \to B$ (baseline lung function), $B \to Y$ (lung function at one year), $X \to Y$ . Both $\{A\}$ and $\{B\}$ are admissible. In the simulation below, which gives the smaller standard error, and why?

Show solution

Step 1. Back-door path: $X \leftarrow A \to B \to Y$ , blocked by $A$ (fork) or by $B$ (chain).

Step 2. $B$ is a parent of $Y$ : controlling for it removes the large outcome variance it explains, shrinking the numerator of the variance formula.

Step 3. $B$ is only a noisy proxy for $A$ , so $\operatorname{Var}(X \mid B) > \operatorname{Var}(X \mid A)$ : more treatment variation is kept, enlarging the denominator.

Step 4. $A$ does the opposite on both counts: it predicts $X$ strongly and $Y$ only through $B$ .

Answer: $\{B\}$ is more precise; the simulation shows an SE ratio of about 0.38. Prefer confounders near the outcome.

Exercise 15 · The Table 2 fallacy

A paper estimates the effect of air pollution $P$ on asthma admissions $Y$ , adjusting for temperature $T$ and income $I$ (both confounders; temperature also affects pollution). Its table reports coefficients for $P$ , $T$ and $I$ . A journalist writes "temperature has no effect on asthma once pollution is accounted for". What is wrong?

Show solution

Step 1. The regression was designed so that $\{T, I\}$ is an admissible set for $P$ .

Step 2. For temperature, pollution is a mediator ( $T \to P \to Y$ ). Conditioning on $P$ blocks that channel, so the $T$ coefficient is at most a direct effect, and possibly confounded by variables not needed for $P$ .

Answer: the $T$ coefficient is not the total effect of temperature; only the coefficient on $P$ has the causal interpretation the design supports.

Verify in Python

import numpy as np

rng = np.random.default_rng(53)
def coef_se(y, X):
    X = np.column_stack([np.ones(len(y))] + X)
    b, *_ = np.linalg.lstsq(X, y, rcond=None); r = y - X @ b
    V = r @ r / (len(y) - X.shape[1]) * np.linalg.inv(X.T @ X)
    return b[1], np.sqrt(V[1, 1])

n = 200_000
# Exercise 7 / 13: bias amplification
D = rng.normal(size=n); A = rng.normal(size=n)
S = .8 * D + A + rng.normal(size=n); Y = .08 * S + .10 * A + rng.normal(size=n)
print("schooling: no control %.4f, control for distance %.4f (truth 0.08)" % (coef_se(Y, [S])[0], coef_se(Y, [S, D])[0]))

# Exercise 9: confounded mediator (a=1, b=1, c=1, d=0.5, vU=vM=1 -> coef 0.5 - 0.5 = 0)
X = rng.normal(size=n); U = rng.normal(size=n)
M = X + U + rng.normal(size=n); Y = .5 * X + M + U + rng.normal(size=n)
print("coef on X given M: %.3f (direct effect 0.5, formula 0.0)" % coef_se(Y, [X, M])[0])

# Exercise 10: M-bias with alpha = beta = 1 (tau = 1)
U1, U2 = rng.normal(size=n), rng.normal(size=n)
X = U1 + rng.normal(size=n); M = U1 + U2 + rng.normal(size=n); Y = X + U2 + rng.normal(size=n)
print("M-bias: without M %.3f, with M %.3f (bias -0.2)" % (coef_se(Y, [X])[0], coef_se(Y, [X, M])[0]))

# Exercise 14: precision of two admissible sets, 400 patients, 2000 replications
est = {"A": [], "B": []}
for _ in range(2000):
    m = 400
    A = rng.normal(size=m); X = (1.5 * A + rng.normal(size=m) > 0).astype(float)
    B = A + rng.normal(size=m); Y = 1.0 * X + 2 * B + rng.normal(size=m)
    est["A"].append(coef_se(Y, [X, A])[0]); est["B"].append(coef_se(Y, [X, B])[0])
sa, sb = np.std(est["A"]), np.std(est["B"])
print(f"mean A {np.mean(est['A']):.3f}, B {np.mean(est['B']):.3f}; SD A {sa:.3f}, B {sb:.3f}, ratio {sb / sa:.2f}")

Key takeaways

Include confounders (or their proxies); add strong outcome predictors for precision.
Leave out instruments, mediators, colliders (including pre-treatment M-colliders) and descendants of the outcome.
Among admissible sets, those closer to the outcome are more precise; coefficients on controls are not causal by default.

Day 53. Adjustment Sets & Bad Controls

Theory

A catalogue of controls

Precision in a linear model

Choosing among admissible sets

The Table 2 fallacy

Part 1 · Warm-up

Exercise 1 · Classify the controls

Exercise 2 · Precision from an outcome predictor

Exercise 3 · Precision lost to an instrument

Exercise 4 · M-bias

Exercise 5 · Controlling for a consequence of the outcome

Part 2 · Proofs

Exercise 6 · The variance formula

Exercise 7 · Bias amplification

Exercise 8 · Outcome predictors do not change the estimand

Exercise 9 · Controlling for a confounded mediator

Exercise 10 · The size of M-bias

Part 3 · Applied

Exercise 11 · Medicaid expansion and mortality

Exercise 12 · A cash-transfer RCT

Exercise 13 · Bias amplification in returns to schooling

Exercise 14 · Choosing the most precise admissible set

Exercise 15 · The Table 2 fallacy

Verify in Python

Key takeaways

Simulation Lab: Good & Bad Controls