Day 15: Phase 1 Capstone: Simpson's Paradox, Conditional Independence & Selection Bias

Theory

Conditional independence

$X$ and $Y$ are conditionally independent given $Z$ , written $X \perp Y \mid Z$ , if

P(X, Y \mid Z) = P(X \mid Z)\,P(Y \mid Z) \quad\Longleftrightarrow\quad P(X \mid Y, Z) = P(X \mid Z).

Two surprises that drive much of causal inference:

Variables can be dependent overall but independent within strata (a common cause $Z$ explains their association).
Variables can be independent overall but dependent within strata (conditioning on a common effect, a "collider", creates an association: Berkson's paradox).

Phase 4 turns these patterns into graphs (DAGs and d-separation). Today we stay with algebra.

Simpson's paradox

By the law of total probability,

P(Y = 1 \mid T = t) = \sum_z P(Y = 1 \mid T = t, Z = z)\,P(Z = z \mid T = t).

The overall rate for each arm averages its stratum rates with its own weights $P(Z \mid T = t)$ . If the arms have different case mixes, the overall comparison can reverse the within-stratum comparison. Standardising uses common weights $P(Z = z)$ instead, which is Day 6's adjustment formula.

Selection bias decomposition

With potential outcomes $Y(1), Y(0)$ and observed $Y = Y(T)$ :

\underbrace{E[Y \mid T = 1] - E[Y \mid T = 0]}_{\text{naive comparison}} = \underbrace{E[Y(1) - Y(0) \mid T = 1]}_{\text{ATT}} + \underbrace{E[Y(0) \mid T = 1] - E[Y(0) \mid T = 0]}_{\text{selection bias}}.

Selection bias is the gap in untreated outcomes between those who did and did not get treatment. It is zero when $T \perp Y(0)$ , which randomisation guarantees. When we can only claim $\big(Y(1), Y(0)\big) \perp T \mid Z$ (unconfoundedness), the decomposition holds within each stratum of $Z$ , and we standardise.

Your Phase 1 toolkit

Question	Tool (Day)
What can I say about a joint probability from marginals?	Fréchet bounds (1)
How do overall and subgroup rates relate?	Law of total probability (2), LIE (6)
Is the effect an average of something?	Linearity of expectation (3)
How does omitting a variable bias a slope?	Covariance algebra (5)
How precise is my estimate?	CLT, SE, CI, bootstrap (7–9)
Could this be noise? Can my study detect it?	Tests and power (10)
How do I fit a model / a propensity score?	Maximum likelihood (11)
SE of a ratio or % effect?	Delta method (12)
How do I combine prior evidence or shrink noisy estimates?	Bayes (13)
Some outcomes are unobserved	MAR, IPW (14)

Part 1 · Warm-up

The classic kidney-stone study (Charig et al., 1986) compared open surgery (A) with percutaneous nephrolithotomy (B):

	Small stones	Large stones	All
A: successes / patients	81 / 87	192 / 263	273 / 350
B: successes / patients	234 / 270	55 / 80	289 / 350

Exercise 1 · Stratum rates

Compute each treatment's success rate for small and large stones.

Show solution

Step 1. Small stones: A $81/87 \approx 93.1\%$ ; B $234/270 \approx 86.7\%$ .

Step 2. Large stones: A $192/263 \approx 73.0\%$ ; B $55/80 \approx 68.8\%$ .

Answer: A has the higher success rate for both stone sizes.

Exercise 2 · Overall rates

Compute the overall success rates.

Show solution

Step 1. A: $273/350 = 78.0\%$ .

Step 2. B: $289/350 \approx 82.6\%$ .

Answer: B looks better overall although A is better for each stone size: Simpson's paradox.

Exercise 3 · The weights

What share of each treatment's patients had large stones? Explain the reversal.

Show solution

Step 1. Share with large stones: A $263/350 \approx 75\%$ ; B $80/350 \approx 23\%$ .

Step 2. Large stones are harder to treat (success ~70% versus ~90%), and A got mostly large stones.

Answer: each overall rate averages that treatment's stratum rates with its own case mix. A's average leans on the difficult stratum, which drags it below B's.

Exercise 4 · Dependent overall, independent within strata

$Z \sim \text{Bern}(0.5)$ . Given $Z$ , $X$ and $Y$ are independent Bernoulli( $0.2 + 0.6Z$ ). Are $X$ and $Y$ independent?

Show solution

Step 1. By construction $X \perp Y \mid Z$ .

Step 2. Marginally, $P(X = 1) = P(Y = 1) = 0.5$ , but $P(X = 1, Y = 1) = 0.5(0.2^2) + 0.5(0.8^2) = 0.34 \ne 0.25$ .

Step 3. They are dependent: the common cause $Z$ links them.

Exercise 5 · Independent overall, dependent within strata

Talent $T$ and charm $C$ are independent Bernoulli(0.5). A firm hires anyone with at least one. Among hires, compute $P(T = 1)$ and $P(T = 1 \mid C = 1)$ .

Show solution

Step 1. Hired = $\{T = 1 \text{ or } C = 1\}$ , probability 0.75.

Step 2. $P(T = 1 \mid \text{hired}) = 0.5/0.75 = 2/3$ .

Step 3. Among hires with charm, talent is 1 with probability $0.25/0.5 = 1/2$ .

Step 4. So among hires, charm and talent are negatively related, although they are independent in the population.

Step 5. Conditioning on a common effect (hiring) created the association.

Part 2 · Proofs

Exercise 6 · The selection-bias decomposition

Prove the decomposition of the naive comparison into ATT plus selection bias.

Show solution

$E[Y \mid T = 1] = E[Y(1) \mid T = 1]$ and $E[Y \mid T = 0] = E[Y(0) \mid T = 0]$ . Add and subtract $E[Y(0) \mid T = 1]$ :

E[Y(1) \mid T = 1] - E[Y(0) \mid T = 1] + E[Y(0) \mid T = 1] - E[Y(0) \mid T = 0].

The first pair is the ATT (by linearity); the second pair is selection bias.

Exercise 7 · Randomisation removes selection bias

Show that if $T \perp \big(Y(1), Y(0)\big)$ , the naive comparison equals the ATE.

Show solution

Independence gives $E[Y(0) \mid T = 1] = E[Y(0) \mid T = 0] = E[Y(0)]$ , so selection bias is zero. It also gives $E[Y(1) - Y(0) \mid T = 1] = E[Y(1) - Y(0)]$ , so ATT = ATE.

Exercise 8 · The adjustment formula

Prove that if $\big(Y(1), Y(0)\big) \perp T \mid Z$ and $0 < P(T = 1 \mid Z) < 1$ , then $E[Y(1)] = \sum_z E[Y \mid T = 1, Z = z]P(Z = z)$ .

Show solution

E[Y(1)] = \sum_z E[Y(1) \mid Z = z]P(z) = \sum_z E[Y(1) \mid T = 1, Z = z]P(z) = \sum_z E[Y \mid T = 1, Z = z]P(z).

The first step is the LIE, the second unconfoundedness, the third that $Y = Y(1)$ when $T = 1$ . Overlap ensures $E[Y \mid T = 1, Z = z]$ is defined for every $z$ .

Exercise 9 · No reversal without imbalance

Show that if $P(Z \mid T = 1) = P(Z \mid T = 0)$ and A is better in every stratum, A is better overall.

Show solution

Step 1. With common weights $w_z$ , $P(Y \mid A) - P(Y \mid B) = \sum_z w_z\big[P(Y \mid A, z) - P(Y \mid B, z)\big]$ , a weighted sum of positive numbers with non-negative weights summing to 1, hence positive.

Step 2. Simpson's reversal needs the case mix to differ between arms, which randomisation prevents in expectation.

Exercise 10 · Two forms of conditional independence

Show that $P(X, Y \mid Z) = P(X \mid Z)P(Y \mid Z)$ implies $P(X \mid Y, Z) = P(X \mid Z)$ (where defined).

Show solution

P(X \mid Y, Z) = \frac{P(X, Y \mid Z)}{P(Y \mid Z)} = \frac{P(X \mid Z)P(Y \mid Z)}{P(Y \mid Z)} = P(X \mid Z).

Once you know $Z$ , learning $Y$ tells you nothing more about $X$ .

Part 3 · Applied

Exercise 11 · Standardising the kidney-stone data

Assume unconfoundedness given stone size and standardise to the pooled population (357 small and 343 large of 700). Which treatment is better, and by how much?

Show solution

Step 1. Common weights from the pooled population: small $357/700 = 0.51$ , large $343/700 = 0.49$ .

Step 2. Standardised success for A:

0.931(0.51) + 0.730(0.49) \approx 0.475 + 0.358 = 0.833.

Step 3. Standardised success for B:

0.867(0.51) + 0.688(0.49) \approx 0.442 + 0.337 = 0.779.

Step 4. Difference: $0.833 - 0.779 \approx 0.054$ .

Answer: with both treatments applied to the same case mix, A succeeds about 5.4 points more often, reversing the naive comparison.

Exercise 12 · Selection bias in job training

Trainees earn $30k on average and non-trainees $34k. A credible study finds the ATT is +$2k. What is the selection bias, and what does it mean?

Show solution

Step 1. Naive comparison: $30 - 34 = -4$ (thousand dollars).

Step 2. Decomposition: naive $=$ ATT $+$ selection bias, so $-4 = 2 + \text{bias}$ .

Step 3. Selection bias $= -6$ .

Answer: −$6k. Without training, trainees would have earned $6k less than non-trainees, because the programme targets disadvantaged workers.

Exercise 13 · Hospital-based studies

Two diseases are independent in the population, but each raises the chance of hospitalisation. A study of hospital patients finds them negatively associated. Explain.

Show solution

Step 1. Draw the structure: disease 1 → hospitalised ← disease 2. Hospitalisation is a common effect (a collider).

Step 2. Among hospitalised patients, someone without disease 1 must have been admitted for another reason, so is more likely to have disease 2.

Step 3. So within the hospital sample the diseases are negatively associated, though independent in the population (Berkson, 1946).

Answer: selecting the sample on a common effect creates a spurious association. Do not condition on variables affected by both the exposure and the outcome.

Exercise 14 · Controlling for a post-treatment variable

A training programme raises both job quality and wages. An analyst "controls for" job quality when estimating the effect on wages. What goes wrong?

Show solution

Step 1. Training → job quality → wages: job quality is a mediator.

Step 2. Holding job quality fixed removes the part of the effect that works through better jobs, so the estimate is no longer the total effect.

Step 3. If unobserved factors (e.g. motivation) affect both job quality and wages, conditioning on job quality also opens a collider path and adds bias.

Answer: control only for pre-treatment variables when estimating a total effect.

Exercise 15 · Choose the tool

For each task, name the Phase 1 tool: (a) a 95% interval for a median income; (b) the SE of a percentage effect from a log-point coefficient; (c) correcting a survey mean when response depends on age; (d) the sample size for 80% power; (e) estimating a propensity score.

Show solution

Step 1. (a) Median income has no simple SE formula → bootstrap (Day 9).

Step 2. (b) A function of an estimate → delta method (Day 12).

Step 3. (c) Response depends on an observed covariate → IPW under MAR (Day 14).

Step 4. (d) Planning a study → power formula (Day 10).

Step 5. (e) A binary treatment model → logit by maximum likelihood (Day 11).

Verify in Python

import numpy as np

succ = {"A": (81, 87, 192, 263), "B": (234, 270, 55, 80)}
w_small = (87 + 270) / 700
for arm, (s1, n1, s2, n2) in succ.items():
    r1, r2 = s1 / n1, s2 / n2
    print(f"{arm}: small {r1:.3f}, large {r2:.3f}, overall {(s1 + s2) / (n1 + n2):.3f}, "
          f"standardised {w_small * r1 + (1 - w_small) * r2:.3f}")

# Berkson's paradox by simulation
rng = np.random.default_rng(15)
talent = rng.random(1_000_000) < 0.5
charm = rng.random(1_000_000) < 0.5
hired = talent | charm
print("corr in population:", np.corrcoef(talent, charm)[0, 1].round(3),
      " corr among hires:", np.corrcoef(talent[hired], charm[hired])[0, 1].round(3))

Key takeaways

A naive comparison = ATT + selection bias; randomisation sets the bias to zero.
Simpson's paradox comes from different case mixes; standardise with common weights.
Conditioning on a common cause can remove an association; conditioning on a common effect can create one.

Day 15. Phase 1 Capstone: Simpson's Paradox, Conditional Independence & Selection Bias

Theory

Conditional independence

Simpson's paradox

Selection bias decomposition

Your Phase 1 toolkit

Part 1 · Warm-up

Exercise 1 · Stratum rates

Exercise 2 · Overall rates

Exercise 3 · The weights

Exercise 4 · Dependent overall, independent within strata

Exercise 5 · Independent overall, dependent within strata

Part 2 · Proofs

Exercise 6 · The selection-bias decomposition

Exercise 7 · Randomisation removes selection bias

Exercise 8 · The adjustment formula

Exercise 9 · No reversal without imbalance

Exercise 10 · Two forms of conditional independence

Part 3 · Applied

Exercise 11 · Standardising the kidney-stone data

Exercise 12 · Selection bias in job training

Exercise 13 · Hospital-based studies

Exercise 14 · Controlling for a post-treatment variable

Exercise 15 · Choose the tool

Verify in Python

Key takeaways

Simulation Lab: Simpson's Paradox Lab