Day 48: Chains, Forks & Colliders · Math for Causal Inference

Theory

Three junctions

Any path through a middle node $M$ looks like one of these:

Name	Shape	Unconditionally	Conditioning on $M$
Chain (mediator)	$A \to M \to B$	$A, B$ associated	association blocked
Fork (confounder)	$A \leftarrow M \to B$	$A, B$ associated	association blocked
Collider	$A \to M \leftarrow B$	$A, B$ independent	association created

Conditioning on a descendant of a collider also opens it, partially.

Intuition

Chain: job training raises skills, skills raise wages. Among workers with the same skills, training tells you nothing more about wages.
Fork: income raises both gym membership and health. Among people with the same income, gym membership and health are unrelated (in this story).
Collider: ability and connections both help land a job at a top firm. Among employees, a worker with weak connections must have had high ability to get in: the two become negatively related.

Linear path analysis

Take standardised variables (mean 0, variance 1) in a linear model where each arrow has a coefficient. Sewall Wright's path-tracing rule: the correlation between two variables is the sum, over all paths without colliders, of the products of the coefficients along the path.

Chain $Z \xrightarrow{a} X \xrightarrow{b} Y$ : $\operatorname{corr}(Z, Y) = ab$ . Fork $Z \xleftarrow{a} X \xrightarrow{b} Y$ : $\operatorname{corr}(Z, Y) = ab$ . Collider $Z \to X \leftarrow Y$ : $\operatorname{corr}(Z, Y) = 0$ .

Conditioning in the Gaussian case

For jointly normal variables, conditioning on $X$ leaves the covariance

\operatorname{Cov}(Z, Y \mid X) = \operatorname{Cov}(Z, Y) - \frac{\operatorname{Cov}(Z, X)\operatorname{Cov}(X, Y)}{\operatorname{Var}(X)},

which is what a regression of $Y$ on $Z$ and $X$ estimates (Frisch–Waugh–Lovell, Day 28). For the collider $X = Z + Y + e$ the first term is 0 and the second is non-zero, so conditioning creates a covariance.

Ways of "conditioning"

Conditioning happens whenever you stratify, add a variable to a regression, match on it, or select the sample on it: studying only hospital patients, only employed workers, only firms that survived, only people who answered a survey. Selection is conditioning you may not notice (Day 57).

Part 1 · Warm-up

Exercise 1 · Name the junction

In the DAG $A \to B \to C$ , $A \to D \leftarrow C$ , $E \to A$ , $E \to F$ , classify: (a) $A, B, C$ ; (b) $A, D, C$ ; (c) $B, A, E$ (on the path $B \leftarrow A \leftarrow E$ ); (d) $A, E, F$ .

Show solution

Step 1. (a) $A \to B \to C$ : chain.

Step 2. (b) $A \to D \leftarrow C$ : collider at $D$ .

Step 3. (c) $B \leftarrow A \leftarrow E$ : arrows run in one direction through $A$ , so a chain (read backwards).

Step 4. (d) $A \leftarrow E \to F$ : fork at $E$ .

Exercise 2 · A chain's correlation

Standardised variables follow $Z \xrightarrow{0.5} X \xrightarrow{0.6} Y$ . Find $\operatorname{corr}(Z, Y)$ .

Show solution

Step 1. One path without colliders: $Z \to X \to Y$ .

Step 2. Multiply the coefficients: $0.5 \times 0.6 = 0.30$ .

Answer: 0.30.

Exercise 3 · A fork's correlation

Standardised variables follow $Z \xleftarrow{0.7} X \xrightarrow{-0.4} Y$ . Find $\operatorname{corr}(Z, Y)$ and the correlation given $X$ .

Show solution

Step 1. One path: $Z \leftarrow X \to Y$ ; product $0.7 \times (-0.4) = -0.28$ .

Step 2. Given $X$ , the fork is blocked: partial correlation 0.

Answer: −0.28 unconditionally; 0 given $X$ .

Exercise 4 · A collider's correlation

$Z$ and $Y$ are independent standard normals and $X = Z + Y$ (no noise). Find $\operatorname{corr}(Z, Y)$ and $\operatorname{corr}(Z, Y \mid X)$ .

Show solution

Step 1. Unconditionally they are independent: correlation 0.

Step 2. Given $X = x$ , $Y = x - Z$ exactly.

Step 3. So $Y$ is a decreasing linear function of $Z$ : correlation −1.

Answer: 0 and −1. Conditioning on a collider can create the strongest possible association.

Exercise 5 · Open or blocked?

For each path, say whether it transmits association: (a) $A \to M \to B$ given $M$ ; (b) $A \to M \leftarrow B$ given nothing; (c) $A \to M \leftarrow B$ given a child of $M$ ; (d) $A \leftarrow M \to B$ given nothing.

Show solution

Step 1. (a) Chain, middle conditioned: blocked.

Step 2. (b) Collider, not conditioned: blocked.

Step 3. (c) Collider with a conditioned descendant: open.

Step 4. (d) Fork, not conditioned: open.

Part 2 · Proofs

Exercise 6 · The linear chain

Let $X = aZ + e_X$ and $Y = bX + e_Y$ , with $Z, e_X, e_Y$ independent, mean 0. Prove $\operatorname{Cov}(Z, Y) = ab\operatorname{Var}(Z)$ and $\operatorname{Cov}(Z, Y \mid X) = 0$ (in the partial-covariance sense).

Show solution

Step 1. Substitute: $Y = abZ + be_X + e_Y$ .

Step 2. $\operatorname{Cov}(Z, Y) = ab\operatorname{Var}(Z)$ , since $Z$ is independent of $e_X, e_Y$ .

Step 3. $\operatorname{Cov}(Z, X) = a\operatorname{Var}(Z)$ and $\operatorname{Cov}(X, Y) = b\operatorname{Var}(X)$ .

Step 4. Partial covariance:

\operatorname{Cov}(Z, Y) - \frac{\operatorname{Cov}(Z, X)\operatorname{Cov}(X, Y)}{\operatorname{Var}(X)} = ab\operatorname{Var}(Z) - \frac{a\operatorname{Var}(Z)\,b\operatorname{Var}(X)}{\operatorname{Var}(X)} = 0. \quad \blacksquare

Exercise 7 · The linear fork

Let $Z = aX + e_Z$ and $Y = bX + e_Y$ with $X, e_Z, e_Y$ independent. Prove $\operatorname{Cov}(Z, Y) = ab\operatorname{Var}(X)$ and the partial covariance given $X$ is 0.

Show solution

Step 1. $\operatorname{Cov}(Z, Y) = \operatorname{Cov}(aX + e_Z, bX + e_Y) = ab\operatorname{Var}(X)$ .

Step 2. $\operatorname{Cov}(Z, X) = a\operatorname{Var}(X)$ and $\operatorname{Cov}(X, Y) = b\operatorname{Var}(X)$ .

Step 3. Partial covariance: $ab\operatorname{Var}(X) - \dfrac{a\operatorname{Var}(X)\,b\operatorname{Var}(X)}{\operatorname{Var}(X)} = 0$ . $\blacksquare$

Exercise 8 · Conditioning on a collider

Let $Z, Y$ be independent $N(0, 1)$ and $X = Z + Y + e$ with $e \sim N(0, \sigma^2)$ independent. Prove that the partial correlation of $Z$ and $Y$ given $X$ is $-1/(1 + \sigma^2)$ .

Show solution

Step 1. $\operatorname{Var}(X) = 2 + \sigma^2$ , $\operatorname{Cov}(Z, X) = \operatorname{Cov}(Y, X) = 1$ , $\operatorname{Cov}(Z, Y) = 0$ .

Step 2. Partial covariance: $0 - \dfrac{1 \cdot 1}{2 + \sigma^2} = -\dfrac{1}{2 + \sigma^2}$ .

Step 3. Partial variances: $\operatorname{Var}(Z \mid X) = \operatorname{Var}(Y \mid X) = 1 - \dfrac{1}{2 + \sigma^2} = \dfrac{1 + \sigma^2}{2 + \sigma^2}$ .

Step 4. Partial correlation:

\frac{-1/(2 + \sigma^2)}{(1 + \sigma^2)/(2 + \sigma^2)} = -\frac{1}{1 + \sigma^2}. \quad \blacksquare

With $\sigma^2 = 1$ it is −0.5; with no noise it is −1 (Exercise 4).

Exercise 9 · Conditioning on a collider's descendant

In Exercise 8 add $W = X + u$ with $u \sim N(0, \tau^2)$ independent. Prove the partial correlation of $Z$ and $Y$ given $W$ is $-1/(1 + \sigma^2 + \tau^2)$ .

Show solution

Step 1. $W = Z + Y + e + u$ , so $\operatorname{Var}(W) = 2 + \sigma^2 + \tau^2$ and $\operatorname{Cov}(Z, W) = \operatorname{Cov}(Y, W) = 1$ .

Step 2. This has exactly the form of Exercise 8 with noise variance $\sigma^2 + \tau^2$ in place of $\sigma^2$ .

Step 3. So the partial correlation is $-1/(1 + \sigma^2 + \tau^2)$ . $\blacksquare$

A noisy descendant opens the collider partially: weaker, but not zero.

Exercise 10 · Regression in a chain

In the chain of Exercise 6, regress $Y$ on both $Z$ and $X$ . Prove that the population coefficient on $Z$ is 0.

Show solution

Step 1. By Frisch–Waugh–Lovell, the coefficient on $Z$ equals $\operatorname{Cov}(\tilde Z, Y)/\operatorname{Var}(\tilde Z)$ , where $\tilde Z$ is $Z$ residualised on $X$ .

Step 2. $\operatorname{Cov}(\tilde Z, Y)$ is the partial covariance of $Z$ and $Y$ given $X$ .

Step 3. By Exercise 6 it is 0, so the coefficient is 0. $\blacksquare$ Controlling for a mediator removes the whole effect that passes through it.

Part 3 · Applied

Exercise 11 · Hiring at a top firm

Applicants' ability $A$ and family connections $C$ are independent standard normals. A firm hires when $A + C > 1.5$ . In simulation (code below), what share is hired, and what is $\operatorname{corr}(A, C)$ among employees? Explain the HR department's finding that "well-connected hires are less able".

Show solution

Step 1. Hiring is a collider: $A \to \text{Hired} \leftarrow C$ .

Step 2. Studying employees conditions on it. Among those hired, low $C$ must be offset by high $A$ to pass the threshold.

Step 3. The simulation gives about 14% hired and a correlation of about −0.68 among them (0 in the applicant pool).

Answer: the negative relation is created by selection; it says nothing about whether connections reduce ability. The firm would see the same pattern even if the two were unrelated in the population.

Exercise 12 · The birth-weight paradox

Maternal smoking $S$ (30% of mothers) and a serious birth defect $D$ (5%) are independent. Both cause low birth weight $L$ : $P(L = 1 \mid S, D)$ is 0.05 (neither), 0.15 (smoking only), 0.6 (defect only), 0.7 (both). Infant mortality is $P(M = 1 \mid L, D) = 0.005 + 0.02L + 0.10D$ ; smoking affects mortality only through birth weight. Compare mortality for babies of smokers and non-smokers (a) overall and (b) among low-birth-weight babies.

Show solution

Step 1. Overall, $P(L = 1 \mid S = 1) = 0.95(0.15) + 0.05(0.7) = 0.1775$ and $P(L = 1 \mid S = 0) = 0.95(0.05) + 0.05(0.6) = 0.0775$ .

Step 2. Overall mortality: $S = 1$ : $0.005 + 0.02(0.1775) + 0.10(0.05) = 0.01355$ ; $S = 0$ : $0.005 + 0.02(0.0775) + 0.005 = 0.01155$ . Smoking raises mortality.

Step 3. Among low-birth-weight babies, use Bayes: $P(D = 1 \mid L = 1, S = 1) = \dfrac{0.05(0.7)}{0.05(0.7) + 0.95(0.15)} = \dfrac{0.035}{0.1775} \approx 0.197$ .

Step 4. $P(D = 1 \mid L = 1, S = 0) = \dfrac{0.05(0.6)}{0.05(0.6) + 0.95(0.05)} = \dfrac{0.03}{0.0775} \approx 0.387$ .

Step 5. Mortality among low-birth-weight babies: $S = 1$ : $0.025 + 0.10(0.197) \approx 0.0447$ ; $S = 0$ : $0.025 + 0.10(0.387) \approx 0.0637$ .

Answer: among low-birth-weight babies, those of smokers die less often. Low birth weight is a collider; a small baby of a non-smoker is more likely to have a defect. Smoking is harmful overall.

Exercise 13 · Controlling for a mediator

In standardised form, job training $T \xrightarrow{0.6} \text{skills } S \xrightarrow{0.5} \text{wage } W$ , with no direct arrow from $T$ to $W$ . An analyst regresses wages on training "controlling for skills" and finds no effect. What is the total effect, and what went wrong?

Show solution

Step 1. Total effect of a one-sd increase in $T$ : $0.6 \times 0.5 = 0.30$ sd of wages (path tracing).

Step 2. Skills is the middle of a chain; conditioning on it blocks the only causal path (Exercise 10).

Answer: the total effect is 0.30, but the regression with skills reports 0. Don't control for mediators when you want the total effect.

Exercise 14 · Gyms and health

In standardised form, income $I \xrightarrow{0.5}$ gym membership $G$ and $I \xrightarrow{0.4}$ health $H$ , with no arrow between $G$ and $H$ . What correlation between gym membership and health will a survey find, with and without controlling for income?

Show solution

Step 1. One open path: the fork $G \leftarrow I \to H$ .

Step 2. $\operatorname{corr}(G, H) = 0.5 \times 0.4 = 0.20$ .

Step 3. Given income, the fork is blocked: partial correlation 0.

Answer: 0.20 without control, 0 with it. Here the confounder must be controlled.

Exercise 15 · Which to control?

For each study, identify the junction and say whether to condition on the middle variable: (a) effect of a cash transfer on child schooling, middle variable household consumption; (b) effect of air pollution on asthma visits, middle variable temperature (affects both); (c) effect of a new drug on recovery, studying only patients who completed follow-up, when both side effects of the drug and poor health cause dropout.

Show solution

Step 1. (a) Transfer $\to$ consumption $\to$ schooling: chain. Do not condition if you want the total effect.

Step 2. (b) Pollution $\leftarrow$ temperature $\to$ asthma: fork (confounder). Condition on temperature.

Step 3. (c) Drug $\to$ dropout $\leftarrow$ poor health $\to$ recovery: dropout is a collider.

Step 4. Analysing completers conditions on it and opens a non-causal path from drug to recovery.

Answer: don't condition in (a) or (c); do condition in (b). For (c), analyse everyone randomised (intention to treat) with methods for missing outcomes (Day 14).

Verify in Python

import numpy as np

def pcorr(a, b, c):
    """Partial correlation of a and b given c (residualise both on c)."""
    C = np.column_stack([np.ones_like(c), c])
    ra = a - C @ np.linalg.lstsq(C, a, rcond=None)[0]
    rb = b - C @ np.linalg.lstsq(C, b, rcond=None)[0]
    return np.corrcoef(ra, rb)[0, 1]

rng = np.random.default_rng(48)
n = 1_000_000
# chain, fork and collider with standardised coefficients
Z = rng.normal(size=n); X = .5 * Z + np.sqrt(1 - .25) * rng.normal(size=n); Y = .6 * X + np.sqrt(1 - .36) * rng.normal(size=n)
print(f"chain:    corr {np.corrcoef(Z, Y)[0, 1]:.3f} (0.30)  given X {pcorr(Z, Y, X):.3f}")
X = rng.normal(size=n); Z = .7 * X + np.sqrt(1 - .49) * rng.normal(size=n); Y = -.4 * X + np.sqrt(1 - .16) * rng.normal(size=n)
print(f"fork:     corr {np.corrcoef(Z, Y)[0, 1]:.3f} (-0.28) given X {pcorr(Z, Y, X):.3f}")
Z = rng.normal(size=n); Y = rng.normal(size=n); X = Z + Y + rng.normal(size=n); W = X + rng.normal(size=n)
print(f"collider: corr {np.corrcoef(Z, Y)[0, 1]:.3f}  given X {pcorr(Z, Y, X):.3f} (-1/2)  given W {pcorr(Z, Y, W):.3f} (-1/3)")

# Exercise 11: hiring
A = rng.normal(size=n); C = rng.normal(size=n); hired = A + C > 1.5
print(f"hired {hired.mean():.3f}; corr(A, C) all {np.corrcoef(A, C)[0, 1]:.3f}, hired {np.corrcoef(A[hired], C[hired])[0, 1]:.3f}")

# Exercise 12: birth-weight paradox (exact)
pL = {(0, 0): .05, (1, 0): .15, (0, 1): .6, (1, 1): .7}          # key (s, d)
mort = lambda l, d: .005 + .02 * l + .10 * d
for s in (0, 1):
    overall = sum((.05 if d else .95) * (pL[s, d] * mort(1, d) + (1 - pL[s, d]) * mort(0, d)) for d in (0, 1))
    pd = .05 * pL[s, 1] / (.05 * pL[s, 1] + .95 * pL[s, 0])
    print(f"smoker={s}: mortality overall {overall:.5f}; P(defect|LBW) {pd:.3f}; mortality | LBW {mort(1, 0) + .10 * pd:.4f}")

Key takeaways

Chains and forks transmit association; conditioning on their middle node blocks it.
Colliders block association; conditioning on them, or on their descendants, or selecting the sample on them, creates it.
In linear models, correlations are sums of products of path coefficients over collider-free paths.

Day 48. Chains, Forks & Colliders

Theory

Three junctions

Intuition

Linear path analysis

Conditioning in the Gaussian case

Ways of "conditioning"

Part 1 · Warm-up

Exercise 1 · Name the junction

Exercise 2 · A chain's correlation

Exercise 3 · A fork's correlation

Exercise 4 · A collider's correlation

Exercise 5 · Open or blocked?

Part 2 · Proofs

Exercise 6 · The linear chain

Exercise 7 · The linear fork

Exercise 8 · Conditioning on a collider

Exercise 9 · Conditioning on a collider's descendant

Exercise 10 · Regression in a chain

Part 3 · Applied

Exercise 11 · Hiring at a top firm

Exercise 12 · The birth-weight paradox

Exercise 13 · Controlling for a mediator

Exercise 14 · Gyms and health

Exercise 15 · Which to control?

Verify in Python

Key takeaways

Simulation Lab: Selecting on the Middle Variable