Day 47: Factorisation & the Markov Property · Math for Causal Inference

Theory

From the chain rule to the DAG

For any ordering of variables, the chain rule of probability gives

P(v_1, \dots, v_p) = \prod_{i=1}^p P(v_i \mid v_1, \dots, v_{i-1}).

A DAG says more: in a topological order, each variable depends on its predecessors only through its parents. The joint distribution then factorises:

P(v_1, \dots, v_p) = \prod_{i=1}^p P\big(v_i \mid \operatorname{pa}(v_i)\big).

A distribution that factorises this way is Markov with respect to the DAG. Each factor $P(v_i \mid \operatorname{pa}(v_i))$ is a conditional probability table (CPT) for discrete variables, or a regression model for continuous ones.

Fewer parameters

With $p$ binary variables, an unrestricted joint distribution needs $2^p - 1$ numbers. Under the factorisation, node $i$ needs one probability for each configuration of its parents, $2^{|\operatorname{pa}(v_i)|}$ , so the total is

\sum_{i=1}^p 2^{|\operatorname{pa}(v_i)|}.

With 20 variables and at most two parents each: at most 80 numbers instead of about a million.

The local Markov property

Factorisation is equivalent to the local Markov property:

V_i \perp\!\!\!\perp \operatorname{nd}(V_i) \setminus \operatorname{pa}(V_i) \;\big|\; \operatorname{pa}(V_i),

where $\operatorname{nd}(V_i)$ are the non-descendants of $V_i$ . "Once you know a variable's direct causes, its other non-effects tell you nothing more about it."

Three building blocks

Structure	Factorisation	Implied independence
Chain $Z \to X \to Y$	$P(z)P(x \mid z)P(y \mid x)$	$Y \perp\!\!\!\perp Z \mid X$
Fork $Z \leftarrow X \to Y$	$P(x)P(z \mid x)P(y \mid x)$	$Y \perp\!\!\!\perp Z \mid X$
Collider $Z \to X \leftarrow Y$	$P(z)P(y)P(x \mid z, y)$	$Y \perp\!\!\!\perp Z$ (but not given $X$ )

Day 48 studies these in depth.

Faithfulness

The Markov property says which independences must hold. A distribution may have extra independences, for instance when two causal channels cancel exactly. Faithfulness assumes it does not: every independence in the data comes from the graph. Causal discovery (Day 98) relies on it.

Testable implications

Each implied independence is a testable implication of the DAG. If data reject $Y \perp\!\!\!\perp Z \mid X$ , the graph is wrong. If data agree, the graph survives, but other graphs may imply the same independences.

Part 1 · Warm-up

For Exercises 3–4 use the chain $Z \to X \to Y$ with binary variables, $P(Z = 1) = 0.4$ , $P(X = 1 \mid Z = 1) = 0.7$ , $P(X = 1 \mid Z = 0) = 0.2$ , $P(Y = 1 \mid X = 1) = 0.6$ , $P(Y = 1 \mid X = 0) = 0.1$ .

Exercise 1 · Writing the factorisation

Write the factorisation for the DAG $A \to B$ , $A \to C$ , $B \to D$ , $C \to D$ , $D \to E$ .

Show solution

Step 1. Parents: $A: \varnothing$ ; $B: A$ ; $C: A$ ; $D: B, C$ ; $E: D$ .

Step 2. One factor per node, conditioning on its parents:

P(a, b, c, d, e) = P(a)\,P(b \mid a)\,P(c \mid a)\,P(d \mid b, c)\,P(e \mid d).

Exercise 2 · Counting parameters

All five variables in Exercise 1 are binary. How many parameters does the factorisation need, compared with an unrestricted joint distribution?

Show solution

Step 1. Unrestricted: $2^5 - 1 = 31$ .

Step 2. Factorised: $2^0 + 2^1 + 2^1 + 2^2 + 2^1 = 1 + 2 + 2 + 4 + 2 = 11$ .

Answer: 11 versus 31.

Exercise 3 · A joint probability

Compute $P(Z = 1, X = 1, Y = 1)$ and $P(Z = 0, X = 1, Y = 0)$ .

Show solution

Step 1. Factorisation: $P(z, x, y) = P(z)P(x \mid z)P(y \mid x)$ .

Step 2. $P(1, 1, 1) = 0.4 \times 0.7 \times 0.6 = 0.168$ .

Step 3. $P(0, 1, 0) = 0.6 \times 0.2 \times (1 - 0.6) = 0.6 \times 0.2 \times 0.4 = 0.048$ .

Exercise 4 · Marginals by summing out

Compute $P(X = 1)$ and $P(Y = 1)$ .

Show solution

Step 1. Sum over $Z$ : $P(X = 1) = 0.4(0.7) + 0.6(0.2) = 0.28 + 0.12 = 0.40$ .

Step 2. Sum over $X$ : $P(Y = 1) = P(X = 1)(0.6) + P(X = 0)(0.1)$ .

Step 3. $P(Y = 1) = 0.40(0.6) + 0.60(0.1) = 0.24 + 0.06 = 0.30$ .

Exercise 5 · Local Markov statements

For the DAG in Exercise 1, write the local Markov independence for $D$ and for $E$ .

Show solution

Step 1. $D$ : parents $\{B, C\}$ ; descendants $\{D, E\}$ ; so the non-descendants other than parents are $\{A\}$ .

Step 2. $D \perp\!\!\!\perp A \mid B, C$ .

Step 3. $E$ : parent $\{D\}$ ; non-descendants other than the parent are $\{A, B, C\}$ .

Step 4. $E \perp\!\!\!\perp \{A, B, C\} \mid D$ .

Part 2 · Proofs

Exercise 6 · The chain screens off

For $Z \to X \to Y$ , prove $P(y \mid x, z) = P(y \mid x)$ from the factorisation.

Show solution

Step 1. By definition $P(y \mid x, z) = \dfrac{P(z, x, y)}{P(z, x)}$ .

Step 2. Numerator: $P(z)P(x \mid z)P(y \mid x)$ .

Step 3. Denominator: sum the numerator over $y$ : $P(z)P(x \mid z)\sum_y P(y \mid x) = P(z)P(x \mid z)$ .

Step 4. The ratio is $P(y \mid x)$ , which does not involve $z$ . $\blacksquare$

Exercise 7 · The fork screens off

For $Z \leftarrow X \to Y$ , prove $Y \perp\!\!\!\perp Z \mid X$ .

Show solution

Step 1. Factorisation: $P(x, y, z) = P(x)P(z \mid x)P(y \mid x)$ .

Step 2. Divide by $P(x)$ : $P(y, z \mid x) = P(z \mid x)\,P(y \mid x)$ .

Step 3. That is the definition of conditional independence. $\blacksquare$

Exercise 8 · The collider separates its causes

For $Z \to X \leftarrow Y$ , prove $Z \perp\!\!\!\perp Y$ marginally.

Show solution

Step 1. Factorisation: $P(z, x, y) = P(z)P(y)P(x \mid z, y)$ .

Step 2. Sum over $x$ : $P(z, y) = P(z)P(y)\sum_x P(x \mid z, y) = P(z)P(y)$ .

Step 3. So $Z$ and $Y$ are independent. $\blacksquare$ (Conditioning on $X$ generally destroys this; see Exercise 14.)

Exercise 9 · The factorisation is a valid distribution

Prove that $\prod_i P(v_i \mid \operatorname{pa}(v_i))$ sums to 1 over all values, for any choice of CPTs.

Show solution

Step 1. Order the nodes topologically, $V_1, \dots, V_p$ . The last node $V_p$ is a parent of no one, so it appears only in its own factor.

Step 2. Summing over $v_p$ first: $\sum_{v_p} P(v_p \mid \operatorname{pa}(v_p)) = 1$ , which removes the last factor.

Step 3. What remains is the same product for $V_1, \dots, V_{p-1}$ , in which $V_{p-1}$ is now last.

Step 4. Repeat until no factors remain: the total is 1. $\blacksquare$

Exercise 10 · Local Markov implies factorisation

Prove that if each $V_i$ satisfies the local Markov property, then the joint distribution factorises over the DAG.

Show solution

Step 1. Take a topological order and apply the chain rule: $P(v) = \prod_i P(v_i \mid v_1, \dots, v_{i-1})$ .

Step 2. The predecessors $\{V_1, \dots, V_{i-1}\}$ contain all parents of $V_i$ and no descendants of $V_i$ (descendants come later in the order).

Step 3. So the predecessors are $\operatorname{pa}(V_i)$ plus some other non-descendants.

Step 4. By local Markov, those other non-descendants can be dropped: $P(v_i \mid v_1, \dots, v_{i-1}) = P(v_i \mid \operatorname{pa}(v_i))$ .

Step 5. Substituting gives the factorisation. $\blacksquare$

Part 3 · Applied

Exercise 11 · Vaccination and age

In a flu season, being older ( $O = 1$ , 30% of the population) affects both vaccination $V$ and infection $I$ , and vaccination affects infection: $O \to V$ , $O \to I$ , $V \to I$ . $P(V = 1 \mid O = 1) = 0.8$ , $P(V = 1 \mid O = 0) = 0.4$ . Infection risks $P(I = 1 \mid V, O)$ : young unvaccinated 0.10, young vaccinated 0.04, old unvaccinated 0.30, old vaccinated 0.12. Compute $P(I = 1)$ , $P(I = 1 \mid V = 1)$ and $P(I = 1 \mid V = 0)$ . Compare the crude risk ratio with the age-specific ones.

Show solution

Step 1. Joint probabilities with $I = 1$ , using $P(o)P(v \mid o)P(I = 1 \mid v, o)$ :

young, vaccinated: $0.7 \times 0.4 \times 0.04 = 0.0112$ ;
young, unvaccinated: $0.7 \times 0.6 \times 0.10 = 0.0420$ ;
old, vaccinated: $0.3 \times 0.8 \times 0.12 = 0.0288$ ;
old, unvaccinated: $0.3 \times 0.2 \times 0.30 = 0.0180$ .

Step 2. $P(I = 1) = 0.0112 + 0.0420 + 0.0288 + 0.0180 = 0.100$ .

Step 3. $P(V = 1) = 0.7(0.4) + 0.3(0.8) = 0.52$ , so $P(I = 1 \mid V = 1) = (0.0112 + 0.0288)/0.52 = 0.04/0.52 \approx 0.077$ .

Step 4. $P(I = 1 \mid V = 0) = (0.0420 + 0.0180)/0.48 = 0.06/0.48 = 0.125$ .

Step 5. Crude risk ratio: $0.077/0.125 \approx 0.62$ . Age-specific: $0.04/0.10 = 0.4$ and $0.12/0.30 = 0.4$ .

Answer: the vaccine cuts risk by 60% in each age group, but the crude comparison suggests only 38%, because the vaccinated are disproportionately old (the fork $V \leftarrow O \to I$ ).

Exercise 12 · A credit-scoring model

A bank models 10 binary borrower characteristics with a DAG in which each node has at most two parents. Give an upper bound on the number of parameters and compare with the unrestricted joint distribution. What is gained, and what is risked?

Show solution

Step 1. Each node needs at most $2^2 = 4$ parameters, so at most $10 \times 4 = 40$ .

Step 2. Unrestricted: $2^{10} - 1 = 1023$ .

Step 3. Gain: each CPT is estimated from many borrowers, so estimates are precise even for rare combinations.

Answer: at most 40 versus 1,023. The risk is that the missing arrows are wrong; the DAG's implied independences should then be tested.

Exercise 13 · Testing a chain with survey data

A survey of 1,000 jobseekers records programme participation $P$ , whether skills improved $S$ and employment $E$ . Counts:

$P$	$S$	$E = 1$	$E = 0$
1	1	210	90
1	0	80	120
0	1	105	45
0	0	140	210

Do the data agree with the chain $P \to S \to E$ (no direct effect of the programme on employment)?

Show solution

Step 1. The chain implies $E \perp\!\!\!\perp P \mid S$ .

Step 2. With improved skills: $P(E = 1 \mid S = 1, P = 1) = 210/300 = 0.70$ and $P(E = 1 \mid S = 1, P = 0) = 105/150 = 0.70$ .

Step 3. Without: $P(E = 1 \mid S = 0, P = 1) = 80/200 = 0.40$ and $P(E = 1 \mid S = 0, P = 0) = 140/350 = 0.40$ .

Step 4. Marginally: $P(E = 1 \mid P = 1) = 290/500 = 0.58$ versus $P(E = 1 \mid P = 0) = 245/500 = 0.49$ .

Answer: consistent with the chain: the programme is associated with employment, but only through skills. (With sampling noise, a formal test such as a stratified chi-square would be used.)

Exercise 14 · Hospital admissions (Berkson)

Flu $F$ (10% of people) and injury $J$ (5%) are independent. Hospital admission $H$ depends on both: $P(H = 1 \mid F, J)$ is 0.01 for neither, 0.3 for flu only, 0.5 for injury only and 0.8 for both. Among admitted patients, compute $P(J = 1)$ , $P(J = 1 \mid F = 1)$ and $P(J = 1 \mid F = 0)$ .

Show solution

Step 1. Joint probabilities with $H = 1$ :

neither: $0.9 \times 0.95 \times 0.01 = 0.00855$ ;
flu only: $0.1 \times 0.95 \times 0.3 = 0.0285$ ;
injury only: $0.9 \times 0.05 \times 0.5 = 0.0225$ ;
both: $0.1 \times 0.05 \times 0.8 = 0.004$ .

Step 2. $P(H = 1) = 0.06355$ .

Step 3. $P(J = 1 \mid H = 1) = (0.0225 + 0.004)/0.06355 \approx 0.417$ .

Step 4. $P(J = 1 \mid H = 1, F = 1) = 0.004/(0.0285 + 0.004) \approx 0.123$ .

Step 5. $P(J = 1 \mid H = 1, F = 0) = 0.0225/(0.00855 + 0.0225) \approx 0.725$ .

Answer: in the population flu and injury are independent, but among hospital patients those with flu are far less likely to be injured. Conditioning on the collider $H$ creates the association.

Exercise 15 · Microfinance testable implications

A DAG for micro-entrepreneurs: education $E \to$ loan $L$ , $E \to$ income $I$ , $L \to$ business start $B$ , $B \to I$ , rainfall $R \to I$ . List the local Markov independences, and say which one you would test first with survey data.

Show solution

Step 1. $E$ : no parents; non-descendants other than itself: $R$ . So $E \perp\!\!\!\perp R$ .

Step 2. $R$ : no parents; non-descendants: $E, L, B$ . So $R \perp\!\!\!\perp \{E, L, B\}$ .

Step 3. $L$ : parent $E$ ; non-descendants other than $E$ : $R$ . So $L \perp\!\!\!\perp R \mid E$ .

Step 4. $B$ : parent $L$ ; non-descendants: $E, R$ . So $B \perp\!\!\!\perp \{E, R\} \mid L$ .

Step 5. $I$ : parents $E, B, R$ ; non-descendant $L$ . So $I \perp\!\!\!\perp L \mid E, B, R$ .

Answer: test $I \perp\!\!\!\perp L \mid E, B, R$ first: it says loans raise income only by starting businesses. Loans used for consumption smoothing or to pay off costlier debt would violate it.

Verify in Python

import itertools
import numpy as np

# Exercises 3-4: chain Z -> X -> Y
pZ = {1: .4, 0: .6}; pX1 = {1: .7, 0: .2}; pY1 = {1: .6, 0: .1}
P = {(z, x, y): pZ[z] * (pX1[z] if x else 1 - pX1[z]) * (pY1[x] if y else 1 - pY1[x])
     for z, x, y in itertools.product([0, 1], repeat=3)}
print("sums to", round(sum(P.values()), 10), " P(1,1,1) =", round(P[1, 1, 1], 3), " P(0,1,0) =", round(P[0, 1, 0], 3))
print("P(X=1) =", round(sum(v for k, v in P.items() if k[1]), 3), " P(Y=1) =", round(sum(v for k, v in P.items() if k[2]), 3))
for x in (0, 1):   # Y independent of Z given X
    print(f"P(Y=1|X={x},Z=z):", [round(P[z, x, 1] / (P[z, x, 0] + P[z, x, 1]), 3) for z in (0, 1)])

# Exercise 11: vaccination and age
pO = {1: .3, 0: .7}; pV1 = {1: .8, 0: .4}; pI1 = {(0, 0): .10, (1, 0): .04, (0, 1): .30, (1, 1): .12}   # key (v, o)
J = {(o, v): pO[o] * (pV1[o] if v else 1 - pV1[o]) * pI1[v, o] for o in (0, 1) for v in (0, 1)}
pV = {v: sum(pO[o] * (pV1[o] if v else 1 - pV1[o]) for o in (0, 1)) for v in (0, 1)}
r1 = (J[0, 1] + J[1, 1]) / pV[1]; r0 = (J[0, 0] + J[1, 0]) / pV[0]
print("P(I=1) =", round(sum(J.values()), 4), " P(I|V=1) =", round(r1, 4), " P(I|V=0) =", round(r0, 4), " crude RR =", round(r1 / r0, 3))

# Exercise 13: survey counts (P, S) -> (E=1, E=0)
c = {(1, 1): (210, 90), (1, 0): (80, 120), (0, 1): (105, 45), (0, 0): (140, 210)}
for (p, s), (a, b) in c.items():
    print(f"P(E=1 | P={p}, S={s}) = {a / (a + b):.2f}")

# Exercise 14: Berkson's bias
pF, pJ = .1, .05; pH = {(0, 0): .01, (1, 0): .3, (0, 1): .5, (1, 1): .8}
H = {(f, j): (pF if f else 1 - pF) * (pJ if j else 1 - pJ) * pH[f, j] for f in (0, 1) for j in (0, 1)}
tot = sum(H.values())
print("P(H=1) =", round(tot, 5), " P(J|H) =", round((H[0, 1] + H[1, 1]) / tot, 3),
      " P(J|H,F=1) =", round(H[1, 1] / (H[1, 0] + H[1, 1]), 3), " P(J|H,F=0) =", round(H[0, 1] / (H[0, 0] + H[0, 1]), 3))

Key takeaways

A DAG factorises the joint distribution into one conditional distribution per node given its parents.
This is equivalent to the local Markov property: each variable is independent of its non-descendants given its parents.
Every implied independence is a testable implication; colliders are the exception that creates dependence when conditioned on.

z	x	y	P(z, x, y)
1	0	0	0.108
1	0	1	0.012
1	1	0	0.112
1	1	1	0.168
0	0	0	0.432
0	0	1	0.048
0	1	0	0.048
0	1	1	0.072

Day 47. Factorisation & the Markov Property

Theory

From the chain rule to the DAG

Fewer parameters

The local Markov property

Three building blocks

Faithfulness

Testable implications

Part 1 · Warm-up

Exercise 1 · Writing the factorisation

Exercise 2 · Counting parameters

Exercise 3 · A joint probability

Exercise 4 · Marginals by summing out

Exercise 5 · Local Markov statements

Part 2 · Proofs

Exercise 6 · The chain screens off

Exercise 7 · The fork screens off

Exercise 8 · The collider separates its causes

Exercise 9 · The factorisation is a valid distribution

Exercise 10 · Local Markov implies factorisation

Part 3 · Applied

Exercise 11 · Vaccination and age

Exercise 12 · A credit-scoring model

Exercise 13 · Testing a chain with survey data

Exercise 14 · Hospital admissions (Berkson)

Exercise 15 · Microfinance testable implications

Verify in Python

Key takeaways

Simulation Lab: Factorisation & Independence