Day 50: Structural Causal Models · Math for Causal Inference

Theory

Definition

A structural causal model (SCM) $\mathcal M = (\mathbf U, \mathbf V, \mathbf F, P(\mathbf U))$ has:

exogenous variables $\mathbf U$ , determined outside the model, with distribution $P(\mathbf U)$ ;
endogenous variables $\mathbf V = (V_1, \dots, V_p)$ ;
one structural equation per endogenous variable,

V_i := f_i\big(\operatorname{pa}(V_i), U_i\big).

The symbol $:=$ is an assignment: $V_i$ is produced by its causes. Unlike an algebraic equation, it cannot be rearranged to "solve for" a cause.

The induced graph

Draw $W \to V_i$ for each $W \in \operatorname{pa}(V_i)$ . If the $U_i$ are mutually independent the model is Markovian, the graph is a DAG, and the distribution of $\mathbf V$ factorises over it (Day 47). If some $U_i, U_j$ are dependent (an omitted common cause), draw a bidirected edge $V_i \leftrightarrow V_j$ : the model is semi-Markovian. A bidirected edge is shorthand for $V_i \leftarrow U \to V_j$ with $U$ unobserved.

Linear SCMs

Stack the equations: $\mathbf V = \mathbf B\mathbf V + \boldsymbol\varepsilon$ , where $B_{ij}$ is the coefficient of $V_j$ in the equation for $V_i$ and $\operatorname{Var}(\boldsymbol\varepsilon) = \boldsymbol\Omega$ (diagonal if Markovian). Solving,

\mathbf V = (\mathbf I - \mathbf B)^{-1}\boldsymbol\varepsilon, \qquad \boldsymbol\Sigma = \operatorname{Var}(\mathbf V) = (\mathbf I - \mathbf B)^{-1}\boldsymbol\Omega(\mathbf I - \mathbf B)^{-\top}.

Because the graph is acyclic, $\mathbf B$ is nilpotent and $(\mathbf I - \mathbf B)^{-1} = \mathbf I + \mathbf B + \mathbf B^2 + \dots$ : entry $(i, j)$ is the total effect of $V_j$ on $V_i$ , the sum over directed paths of the products of their coefficients.

Structural versus regression coefficients

In $Y := \beta X + \gamma A + U_Y$ , $\beta$ is causal: change $X$ by one unit, holding $A$ and $U_Y$ fixed, and $Y$ moves by $\beta$ . The regression slope of $Y$ on $X$ alone is

\frac{\operatorname{Cov}(X, Y)}{\operatorname{Var}(X)} = \beta + \gamma\,\frac{\operatorname{Cov}(X, A)}{\operatorname{Var}(X)},

which equals $\beta$ only if the omitted cause $A$ is uncorrelated with $X$ . The SCM tells you which regression (if any) recovers $\beta$ .

Observational equivalence

Different SCMs can generate the same distribution. With Gaussian variables, $X \to Y$ and $Y \to X$ fit any correlation equally well. Data alone cannot decide causal direction there; assumptions (time order, experiments, non-Gaussianity, Day 98) must.

What an SCM adds

An SCM supports three kinds of questions (Pearl's ladder, Day 59): association (from $P(\mathbf V)$ ), intervention (replace an equation, Day 51) and counterfactuals (keep each unit's $U$ , Day 59).

Part 1 · Warm-up

Exercise 1 · From equations to graph

Draw the graph for $A := U_A$ , $S := f_S(A, U_S)$ , $O := f_O(S, U_O)$ , $W := f_W(A, S, O, U_W)$ , where $U_S$ and $U_W$ are correlated and the others independent.

Show solution

Step 1. Directed edges from the arguments: $A \to S$ , $S \to O$ , $A \to W$ , $S \to W$ , $O \to W$ .

Step 2. Correlated disturbances $U_S, U_W$ : bidirected edge $S \leftrightarrow W$ .

Answer: a semi-Markovian graph; the bidirected edge says an unobserved factor (say, motivation) affects both schooling and wages.

Exercise 2 · Mean and variance

$X := U_X$ with $U_X \sim N(2, 1)$ and $Y := 3 + 0.5X + U_Y$ with $U_Y \sim N(0, 0.75)$ independent. Find $E[Y]$ , $\operatorname{Var}(Y)$ and $\operatorname{corr}(X, Y)$ .

Show solution

Step 1. $E[Y] = 3 + 0.5(2) = 4$ .

Step 2. $\operatorname{Var}(Y) = 0.5^2(1) + 0.75 = 1$ .

Step 3. $\operatorname{Cov}(X, Y) = 0.5\operatorname{Var}(X) = 0.5$ , so $\operatorname{corr} = 0.5/\sqrt{1 \times 1} = 0.5$ .

Exercise 3 · Total effect by paths

In a linear SCM, $X \xrightarrow{2} M \xrightarrow{3} Y$ and $X \xrightarrow{-1} Y$ . Find the total effect of $X$ on $Y$ .

Show solution

Step 1. Directed paths: $X \to M \to Y$ with product $2 \times 3 = 6$ ; $X \to Y$ with $-1$ .

Step 2. Sum: $6 - 1 = 5$ .

Answer: 5.

Exercise 4 · The covariance matrix

For $X := \varepsilon_X$ , $Y := 2X + \varepsilon_Y$ with $\operatorname{Var}(\varepsilon_X) = 1$ and $\operatorname{Var}(\varepsilon_Y) = 3$ , compute $\boldsymbol\Sigma$ using $(\mathbf I - \mathbf B)^{-1}$ .

Show solution

Step 1. With order $(X, Y)$ : $\mathbf B = \begin{pmatrix} 0 & 0 \\ 2 & 0 \end{pmatrix}$ , so $(\mathbf I - \mathbf B)^{-1} = \mathbf I + \mathbf B = \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix}$ .

Step 2. $\boldsymbol\Omega = \operatorname{diag}(1, 3)$ .

Step 3.

\boldsymbol\Sigma = \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix}\begin{pmatrix} 1 & 0 \\ 0 & 3 \end{pmatrix}\begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 2 \\ 2 & 7 \end{pmatrix}.

Answer: $\operatorname{Var}(X) = 1$ , $\operatorname{Cov}(X, Y) = 2$ , $\operatorname{Var}(Y) = 4 + 3 = 7$ .

Exercise 5 · Assignment, not equation

From $Y := 2X + U_Y$ , a student writes $X = (Y - U_Y)/2$ and concludes that raising $Y$ raises $X$ . What is wrong?

Show solution

Step 1. The structural equation describes how $Y$ is produced; $X$ has its own equation, $X := f_X(\operatorname{pa}(X), U_X)$ , which does not involve $Y$ .

Step 2. Intervening on $Y$ replaces $Y$ 's equation and leaves $X$ 's untouched (Day 51).

Answer: algebraic rearrangement is valid for the observed values but not for causal claims; $X$ does not respond to changes in $Y$ .

Part 2 · Proofs

Exercise 6 · Markovian SCMs factorise

Prove that if the $U_i$ are independent and the graph is acyclic, then $P(\mathbf v) = \prod_i P(v_i \mid \operatorname{pa}(v_i))$ .

Show solution

Step 1. Order the variables topologically. By recursive substitution, each predecessor $V_j$ ( $j < i$ ) is a function of $U_1, \dots, U_{i-1}$ only.

Step 2. $U_i$ is independent of $(U_1, \dots, U_{i-1})$ , hence of the predecessors $(V_1, \dots, V_{i-1})$ .

Step 3. So given the predecessors, $V_i = f_i(\operatorname{pa}(V_i), U_i)$ has a distribution that depends only on $\operatorname{pa}(V_i)$ : $P(v_i \mid v_1, \dots, v_{i-1}) = P(v_i \mid \operatorname{pa}(v_i))$ .

Step 4. Substitute into the chain rule. $\blacksquare$

Exercise 7 · Total effects from (I − B)⁻¹

Prove that for an acyclic linear SCM, $(\mathbf I - \mathbf B)^{-1} = \sum_{k=0}^{p-1}\mathbf B^k$ and that entry $(i, j)$ for $i \ne j$ is the sum over directed paths from $V_j$ to $V_i$ of the products of coefficients.

Show solution

Step 1. $\mathbf B$ is nilpotent ( $\mathbf B^p = 0$ , Day 46), so $(\mathbf I - \mathbf B)\sum_{k=0}^{p-1}\mathbf B^k = \mathbf I - \mathbf B^p = \mathbf I$ .

Step 2. $(\mathbf B^k)_{ij} = \sum B_{i m_1}B_{m_1 m_2}\cdots B_{m_{k-1} j}$ over intermediate nodes: each non-zero term is a directed path $V_j \to V_{m_{k-1}} \to \dots \to V_i$ of length $k$ , weighted by its coefficients.

Step 3. Summing over $k$ adds all directed paths. $\blacksquare$

Exercise 8 · Omitted-variable bias

Let $Y := \beta X + \gamma A + U_Y$ with $U_Y$ independent of $(X, A)$ . Prove $\operatorname{Cov}(X, Y)/\operatorname{Var}(X) = \beta + \gamma\operatorname{Cov}(X, A)/\operatorname{Var}(X)$ .

Show solution

Step 1. $\operatorname{Cov}(X, Y) = \beta\operatorname{Var}(X) + \gamma\operatorname{Cov}(X, A) + \operatorname{Cov}(X, U_Y)$ .

Step 2. The last term is 0 by independence.

Step 3. Divide by $\operatorname{Var}(X)$ . $\blacksquare$

Exercise 9 · Reversing a Gaussian arrow

Let $X \sim N(0, 1)$ and $Y := bX + U_Y$ , $U_Y \sim N(0, 1 - b^2)$ , $|b| < 1$ . Construct an SCM with $Y \to X$ that generates the same joint distribution.

Show solution

Step 1. The original model gives $\operatorname{Var}(Y) = b^2 + (1 - b^2) = 1$ and $\operatorname{Cov}(X, Y) = b$ .

Step 2. Try $Y := U'_Y \sim N(0, 1)$ and $X := bY + U'_X$ with $U'_X \sim N(0, 1 - b^2)$ independent.

Step 3. Then $\operatorname{Var}(X) = b^2 + 1 - b^2 = 1$ and $\operatorname{Cov}(X, Y) = b$ .

Step 4. Both are bivariate normal with mean 0 and the same covariance matrix, so the distributions coincide. $\blacksquare$

Exercise 10 · When regression is structural

In a Markovian linear SCM, prove that regressing $V_i$ on all its parents recovers the structural coefficients.

Show solution

Step 1. $V_i = \sum_{j \in \operatorname{pa}(i)} B_{ij}V_j + \varepsilon_i$ .

Step 2. Each parent is a function of the disturbances of its own ancestors, and $V_i$ is not among them (acyclicity). With independent disturbances, $\varepsilon_i$ is independent of all parents.

Step 3. So $E[\varepsilon_i \mid \operatorname{pa}(V_i)] = 0$ : the OLS moment condition holds at the true coefficients, and the population regression coefficients equal $B_{ij}$ . $\blacksquare$

Part 3 · Applied

Exercise 11 · Returns to schooling with ability

An SCM for log wages: ability $A \sim N(0, 1)$ ; years of schooling $S := 12 + 2A + U_S$ with $\operatorname{Var}(U_S) = 4$ ; $\log W := 1.5 + 0.08S + 0.10A + U_W$ , all disturbances independent. What does a regression of $\log W$ on $S$ alone estimate? What about on $S$ and $A$ ?

Show solution

Step 1. $\operatorname{Var}(S) = 2^2(1) + 4 = 8$ and $\operatorname{Cov}(S, A) = 2$ .

Step 2. By Exercise 8, the short-regression slope is $0.08 + 0.10 \times 2/8 = 0.08 + 0.025 = 0.105$ .

Step 3. With $A$ included, the regression is on all parents of $\log W$ , so it recovers $0.08$ (Exercise 10).

Answer: 10.5% per year without ability, 8% with it. The 2.5-point gap is ability bias; in practice $A$ is unobserved, which motivates instruments (Phase 5).

Exercise 12 · A binary vaccine model

$U_I, U_V, U_E$ are independent uniform on $(0, 1)$ . Exposure $I := \mathbf 1[U_I < 0.1]$ , vaccination $V := \mathbf 1[U_V < 0.6]$ , protection $E := \mathbf 1[U_E < 0.9]$ (the vaccine works for 90%), disease $D := I\,(1 - VE)$ . Compute $P(D = 1)$ , $P(D = 1 \mid V = 1)$ and $P(D = 1 \mid V = 0)$ .

Show solution

Step 1. $D = 1$ requires exposure and no protection: $P(D = 1) = P(I = 1)\big(1 - P(V = 1)P(E = 1)\big) = 0.1(1 - 0.6 \times 0.9) = 0.046$ .

Step 2. Given $V = 1$ : $P(D = 1 \mid V = 1) = 0.1 \times (1 - 0.9) = 0.01$ .

Step 3. Given $V = 0$ : $P(D = 1 \mid V = 0) = 0.1$ .

Answer: 4.6%, 1% and 10%. Because $V$ is independent of $U_I$ and $U_E$ here, the observed risk ratio 0.1 equals the causal one.

Exercise 13 · Monetary policy multipliers

A linear macro SCM: interest rate $R$ affects investment ( $R \xrightarrow{-2} I$ ) and consumption ( $R \xrightarrow{-0.5} C$ ); investment raises consumption ( $I \xrightarrow{0.3} C$ , via income); output $Y := I + C + G$ . Compute the total effect of $R$ on $Y$ by paths and by $(\mathbf I - \mathbf B)^{-1}$ .

Show solution

Step 1. Directed paths $R \to Y$ : $R \to I \to Y$ : $-2 \times 1 = -2$ ; $R \to C \to Y$ : $-0.5 \times 1 = -0.5$ ; $R \to I \to C \to Y$ : $-2 \times 0.3 \times 1 = -0.6$ .

Step 2. Total: $-2 - 0.5 - 0.6 = -3.1$ .

Step 3. With order $(R, I, C, Y)$ , the matrix computation (code below) gives the entry $[(\mathbf I - \mathbf B)^{-1}]_{Y, R} = -3.1$ .

Answer: a one-point rise in the policy rate lowers output by 3.1 units; the regression of $Y$ on $R$ alone recovers this only if $R$ is set independently of the disturbances.

Exercise 14 · Exercise and fitness

A health survey finds $\operatorname{corr}(\text{exercise hours}, \text{fitness score}) = 0.6$ , both standardised and normal. Write two SCMs, one with exercise causing fitness and one with fitness causing exercise, that both fit the data exactly. What evidence could separate them?

Show solution

Step 1. Model 1: $X := U_X \sim N(0, 1)$ , $Y := 0.6X + U_Y$ with $\operatorname{Var}(U_Y) = 1 - 0.36 = 0.64$ .

Step 2. Model 2: $Y := U'_Y \sim N(0, 1)$ , $X := 0.6Y + U'_X$ with $\operatorname{Var}(U'_X) = 0.64$ .

Step 3. By Exercise 9 both give the same bivariate normal distribution.

Answer: only extra information can separate them: an experiment (randomise an exercise programme), time order in panel data (exercise this month, fitness next month), or non-Gaussian features (Day 98).

Exercise 15 · Making a bidirected edge explicit

A health economist models hospital quality $Q$ and patient survival $Y$ with $Q \to Y$ , and notes that wealthier areas have both better hospitals and healthier patients, but area wealth is not measured. Write a semi-Markovian SCM and show what the regression of $Y$ on $Q$ estimates in the linear case $Q := aH + U_Q$ , $Y := \beta Q + cH + U_Y$ , with unobserved wealth $H \sim N(0, 1)$ .

Show solution

Step 1. Graph: $Q \to Y$ and $Q \leftrightarrow Y$ (shorthand for $Q \leftarrow H \to Y$ with $H$ unobserved).

Step 2. $\operatorname{Var}(Q) = a^2 + \operatorname{Var}(U_Q)$ and $\operatorname{Cov}(Q, H) = a$ .

Step 3. By Exercise 8 the slope is $\beta + c\,a/(a^2 + \operatorname{Var}(U_Q))$ .

Answer: biased upward when wealth raises both quality and survival ( $a, c > 0$ ). Days 52–56 ask when such bias can be removed with the variables you do observe.

Verify in Python

import numpy as np

def implied_cov(B, Omega):
    """Covariance of V = B V + eps for an acyclic linear SCM."""
    T = np.linalg.inv(np.eye(len(B)) - B)
    return T @ Omega @ T.T, T

# Exercise 4
S, _ = implied_cov(np.array([[0, 0], [2, 0.]]), np.diag([1, 3.]))
print("Sigma:\n", S)

# Exercise 11: ability bias (order A, S, logW)
B = np.zeros((3, 3)); B[1, 0] = 2; B[2, 1] = .08; B[2, 0] = .10
S, _ = implied_cov(B, np.diag([1, 4, .25]))
print("short slope:", round(S[1, 2] / S[1, 1], 4))
coef = np.linalg.solve(S[:2, :2], S[:2, 2])
print("long regression (A, S):", coef.round(4))

# Exercise 13: total effects, order R, I, C, Y
B = np.zeros((4, 4)); B[1, 0] = -2; B[2, 0] = -.5; B[2, 1] = .3; B[3, 1] = 1; B[3, 2] = 1
_, T = implied_cov(B, np.eye(4))
print("total effect R -> Y:", T[3, 0].round(3))

# Exercise 12: binary vaccine model by simulation
rng = np.random.default_rng(50); n = 2_000_000
I = rng.random(n) < .1; V = rng.random(n) < .6; E = rng.random(n) < .9
D = I & ~(V & E)
print(f"P(D) {D.mean():.4f}  P(D|V=1) {D[V].mean():.4f}  P(D|V=0) {D[~V].mean():.4f}")

# Exercise 14: two directions, one distribution
X = rng.normal(size=n); Y = .6 * X + np.sqrt(.64) * rng.normal(size=n)
Y2 = rng.normal(size=n); X2 = .6 * Y2 + np.sqrt(.64) * rng.normal(size=n)
print("cov X->Y model:", np.cov(X, Y).round(3).tolist(), " cov Y->X model:", np.cov(X2, Y2).round(3).tolist())

Key takeaways

An SCM assigns each variable a mechanism $V_i := f_i(\operatorname{pa}(V_i), U_i)$ ; independent disturbances give a DAG, dependent ones bidirected edges.
In linear SCMs, $(\mathbf I - \mathbf B)^{-1}$ collects total effects and $\boldsymbol\Sigma$ follows from it; regression recovers a structural coefficient only when the regressors block every non-causal path.
Data can be silent about direction: different SCMs can be observationally equivalent.

Day 50. Structural Causal Models

Theory

Definition

The induced graph

Linear SCMs

Structural versus regression coefficients

Observational equivalence

What an SCM adds

Part 1 · Warm-up

Exercise 1 · From equations to graph

Exercise 2 · Mean and variance

Exercise 3 · Total effect by paths

Exercise 4 · The covariance matrix

Exercise 5 · Assignment, not equation

Part 2 · Proofs

Exercise 6 · Markovian SCMs factorise

Exercise 7 · Total effects from (I − B)⁻¹

Exercise 8 · Omitted-variable bias

Exercise 9 · Reversing a Gaussian arrow

Exercise 10 · When regression is structural

Part 3 · Applied

Exercise 11 · Returns to schooling with ability

Exercise 12 · A binary vaccine model

Exercise 13 · Monetary policy multipliers

Exercise 14 · Exercise and fitness

Exercise 15 · Making a bidirected edge explicit

Verify in Python

Key takeaways

Simulation Lab: A Structural Model of Wages