{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 59: Counterfactuals & the Ladder of Causation\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "The top rung: what would have happened to this unit, given what actually happened? Abduction, action and prediction compute counterfactuals from an SCM; probabilities of necessity and sufficiency answer attribution questions, and are only partly identified from data.\n", "\n", "**Learning outcomes**\n", "\n", "- Place a question on the ladder of causation (association, intervention, counterfactual).\n", "- Compute unit-level counterfactuals with abduction, action and prediction.\n", "- Derive and apply the probability of necessity under exogeneity and monotonicity, and bounds on the probability of necessity and sufficiency.\n", "- Show that two models can agree on every experiment and still disagree on counterfactuals." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### The ladder of causation\n", "\n", "| Rung | Question | Notation | Needs |\n", "|---|---|---|---|\n", "| 1. Association | What do I expect to see if I observe $x$? | $P(y \\mid x)$ | data |\n", "| 2. Intervention | What happens if I *do* $x$? | $P(y \\mid \\operatorname{do}(x))$ | data + causal graph (or an experiment) |\n", "| 3. Counterfactual | Given that $x$ and $y$ happened, what would $y$ have been under $x'$? | $P(y_{x'} \\mid x, y)$ | an SCM (functions, not only a graph) |\n", "\n", "Each rung needs more assumptions than the one below.\n", "\n", "### Counterfactuals in an SCM\n", "\n", "For unit $u$ (a value of the disturbances $\\mathbf U$), the counterfactual $Y_x(u)$ is the value of $Y$ in the modified model $\\mathcal M_x$. These are Phase 3's potential outcomes, now derived from structural equations. **Consistency**: if $X(u) = x$ then $Y_x(u) = Y(u)$.\n", "\n", "### Three steps\n", "\n", "To compute $P(y_{x'} \\mid \\text{evidence } e)$:\n", "\n", "1. **Abduction:** update the disturbances using the evidence: $P(\\mathbf u) \\to P(\\mathbf u \\mid e)$.\n", "2. **Action:** replace $X$'s equation by $X := x'$.\n", "3. **Prediction:** compute $Y$ in the modified model, using $P(\\mathbf u \\mid e)$.\n", "\n", "### Linear models: the same effect for everyone\n", "\n", "If $Y := \\beta X + U_Y$ (whatever the dependence between $X$ and $U_Y$), then $Y_{x'}(u) - Y_x(u) = \\beta(x' - x)$ for every unit. Unit-level counterfactuals are easy; they are hard precisely when effects vary across units.\n", "\n", "### Attribution: necessity and sufficiency\n", "\n", "For binary $X, Y$ (write $y$ for $Y = 1$, $y'$ for $Y = 0$):\n", "\n", "- **Probability of necessity:** $PN = P(Y_0 = 0 \\mid X = 1, Y = 1)$: among exposed people who got the disease, the share who would not have without exposure (\"but-for\" causation in law).\n", "- **Probability of sufficiency:** $PS = P(Y_1 = 1 \\mid X = 0, Y = 0)$.\n", "- **Probability of necessity and sufficiency:** $PNS = P(Y_1 = 1, Y_0 = 0)$: the share of units who respond.\n", "\n", "### Identification under extra assumptions\n", "\n", "If $X$ is **exogenous** (randomised) and the effect is **monotone** (no one is harmed: $Y_1 \\ge Y_0$), then\n", "\n", "$$\n", "PN = \\frac{P(y \\mid x) - P(y \\mid x')}{P(y \\mid x)} = 1 - \\frac{1}{RR}, \\qquad PNS = P(y \\mid x) - P(y \\mid x').\n", "$$\n", "\n", "Without monotonicity, experiments give only **bounds**, for example\n", "\n", "$$\n", "\\max\\{0,\\; P(y_x) - P(y_{x'})\\} \\le PNS \\le \\min\\{P(y_x),\\; P(y'_{x'})\\}.\n", "$$\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Which rung?\n", "\n", "Classify: (a) \"Patients who take aspirin have fewer heart attacks.\" (b) \"If we give everyone aspirin, heart attacks will fall by 10%.\" (c) \"Would Mr Lee have avoided his heart attack if he had taken aspirin?\" (d) \"What share of heart attacks among non-users would aspirin have prevented?\"\n", "\n", "
Show solution\n", "\n", "**Step 1.** (a) Rung 1 (association).\n", "\n", "**Step 2.** (b) Rung 2 (intervention).\n", "\n", "**Step 3.** (c) Rung 3: a counterfactual about a specific person, given what happened.\n", "\n", "**Step 4.** (d) Rung 3: a counterfactual conditioned on observed status (non-users who had attacks).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Abduction, action, prediction\n", "\n", "$Y := 2X + U_Y$. A unit has $X = 3$, $Y = 7$. What would $Y$ have been with $X = 5$?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Abduction: $U_Y = Y - 2X = 7 - 6 = 1$.\n", "\n", "**Step 2.** Action: set $X := 5$.\n", "\n", "**Step 3.** Prediction: $Y_5 = 2(5) + 1 = 11$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · Counterfactuals downstream\n", "\n", "$X := U_X$, $M := 0.5X + U_M$, $Y := M + X + U_Y$. A unit has $X = 2$, $M = 1.5$, $Y = 4$. Compute $Y_{X = 0}$ and $M_{X = 0}$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Abduction: $U_M = 1.5 - 1 = 0.5$; $U_Y = 4 - 1.5 - 2 = 0.5$.\n", "\n", "**Step 2.** Action: $X := 0$.\n", "\n", "**Step 3.** Prediction: $M_0 = 0 + 0.5 = 0.5$; $Y_0 = 0.5 + 0 + 0.5 = 1$.\n", "\n", "**Answer:** $M_0 = 0.5$ and $Y_0 = 1$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · Probability of necessity\n", "\n", "In a randomised trial, $P(\\text{recover} \\mid \\text{drug}) = 0.8$ and $P(\\text{recover} \\mid \\text{placebo}) = 0.5$. Assuming no one is harmed by the drug, among treated patients who recovered, what share recovered *because* of the drug?\n", "\n", "
Show solution\n", "\n", "**Step 1.** $PN = \\dfrac{0.8 - 0.5}{0.8} = 0.375$.\n", "\n", "**Answer:** 37.5%; the other 62.5% would have recovered anyway.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Effect of treatment on the treated\n", "\n", "Write the ATT as a counterfactual quantity and say which rung it belongs to.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $ATT = E[Y_1 - Y_0 \\mid X = 1] = E[Y \\mid X = 1] - E[Y_0 \\mid X = 1]$, using consistency.\n", "\n", "**Step 2.** $E[Y_0 \\mid X = 1]$ conditions a counterfactual on the actual treatment.\n", "\n", "**Answer:** rung 3 in form, though it is identified by the same assumptions as rung-2 quantities (back-door, Exercise 14).\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · Consistency from the SCM\n", "\n", "Prove that if $X(u) = x$ then $Y_x(u) = Y(u)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\mathcal M_x$ differs from $\\mathcal M$ only in $X$'s equation.\n", "\n", "**Step 2.** For unit $u$, the original equation already produces $X = x$, so replacing it with $X := x$ changes no value.\n", "\n", "**Step 3.** All other equations and disturbances are identical, so every variable, including $Y$, takes the same value. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · Linear counterfactuals\n", "\n", "Let $Y := \\beta X + U_Y$, with $U_Y$ arbitrarily dependent on $X$. Prove $E[Y_{x'} \\mid X = x, Y = y] = y + \\beta(x' - x)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Abduction: given $X = x$, $Y = y$, we know $U_Y = y - \\beta x$ exactly.\n", "\n", "**Step 2.** Action and prediction: $Y_{x'} = \\beta x' + U_Y = \\beta x' + y - \\beta x$.\n", "\n", "**Step 3.** This is non-random given the evidence, so its expectation is itself. $\\blacksquare$ (Rung 3 collapses to $\\beta$, a rung-2 quantity, in linear additive models.)\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Probability of necessity under exogeneity and monotonicity\n", "\n", "Assume $X \\perp\\!\\!\\!\\perp (Y_0, Y_1)$ and $Y_1 \\ge Y_0$. Prove $PN = [P(y \\mid x) - P(y \\mid x')]/P(y \\mid x)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $PN = \\dfrac{P(Y_0 = 0, X = 1, Y = 1)}{P(X = 1, Y = 1)} = \\dfrac{P(Y_0 = 0, Y_1 = 1, X = 1)}{P(Y_1 = 1, X = 1)}$ (consistency).\n", "\n", "**Step 2.** By exogeneity, $X$ factors out: $PN = P(Y_0 = 0, Y_1 = 1)/P(Y_1 = 1)$.\n", "\n", "**Step 3.** Monotonicity: $Y_1 = 1$ whenever $Y_0 = 1$, so $P(Y_0 = 0, Y_1 = 1) = P(Y_1 = 1) - P(Y_0 = 1)$.\n", "\n", "**Step 4.** By exogeneity $P(Y_x = 1) = P(y \\mid x)$. Substitute. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Bounds on PNS\n", "\n", "From experimental data alone, prove $\\max\\{0, P(y_x) - P(y_{x'})\\} \\le PNS \\le \\min\\{P(y_x), P(y'_{x'})\\}$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Four response types: $(Y_1, Y_0) \\in \\{(1, 1), (1, 0), (0, 1), (0, 0)\\}$ with probabilities $p_{11}, p_{10}, p_{01}, p_{00}$; $PNS = p_{10}$.\n", "\n", "**Step 2.** Experiments give $P(y_x) = p_{11} + p_{10}$ and $P(y_{x'}) = p_{11} + p_{01}$.\n", "\n", "**Step 3.** Upper: $p_{10} \\le p_{11} + p_{10} = P(y_x)$ and $p_{10} \\le p_{10} + p_{00} = 1 - P(y_{x'}) = P(y'_{x'})$.\n", "\n", "**Step 4.** Lower: $p_{10} = P(y_x) - P(y_{x'}) + p_{01} \\ge P(y_x) - P(y_{x'})$, and $p_{10} \\ge 0$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Same experiments, different counterfactuals\n", "\n", "Construct two SCMs for binary $X, Y$ with the same $P(y \\mid \\operatorname{do}(x))$ for both $x$ but different $PNS$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Model A: $U \\sim \\text{Bernoulli}(0.5)$, $Y := U$. Then $P(Y = 1 \\mid \\operatorname{do}(x)) = 0.5$ for both $x$, and nobody responds: $PNS = 0$.\n", "\n", "**Step 2.** Model B: $Y := X \\oplus U$ (exclusive or). Again $P(Y = 1 \\mid \\operatorname{do}(x)) = 0.5$ for both $x$.\n", "\n", "**Step 3.** In Model B, units with $U = 0$ have $Y_1 = 1, Y_0 = 0$ (helped) and units with $U = 1$ have $Y_1 = 0, Y_0 = 1$ (harmed): $PNS = 0.5$.\n", "\n", "**Answer:** identical at rungs 1 and 2, different at rung 3. $\\blacksquare$ A zero average effect can hide large offsetting individual effects.\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · A worker's counterfactual wage\n", "\n", "In Day 50's SCM, $\\log W := 1.5 + 0.08S + 0.10A + U_W$, a worker has 12 years of schooling and $\\log W = 2.6$. What would her log wage have been with 16 years? Does the answer depend on her (unobserved) ability?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Abduction: $0.10A + U_W = 2.6 - 1.5 - 0.08(12) = 0.14$. We cannot separate $A$ from $U_W$, but we do not need to.\n", "\n", "**Step 2.** Action: $S := 16$. Ability is upstream of schooling, so it is unchanged.\n", "\n", "**Step 3.** Prediction: $\\log W_{16} = 1.5 + 0.08(16) + 0.14 = 2.92$.\n", "\n", "**Answer:** 2.92, about 38% higher wages ($e^{0.32} - 1$). Her ability does not matter because the model is linear and additive (Exercise 7); with heterogeneous returns it would.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · But-for causation in court\n", "\n", "A worker exposed to asbestos develops mesothelioma and sues. A cohort study (exposure as good as random given the covariates) gives a lifetime risk of 2.0% for exposed and 0.5% for comparable unexposed workers. Assuming asbestos never prevents the disease, is it \"more likely than not\" that the exposure caused this worker's disease?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Exogeneity and monotonicity give $PN = (0.020 - 0.005)/0.020 = 0.75$.\n", "\n", "**Step 2.** Equivalently, $RR = 4$ and $PN = 1 - 1/4 = 0.75$.\n", "\n", "**Answer:** 75% > 50%: the civil standard is met. The rule \"RR > 2 implies PN > 0.5\" holds under these assumptions; without monotonicity only bounds are available.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · How many people does a vaccine protect?\n", "\n", "A trial finds infection risks of 2% with vaccine and 10% with placebo. Let $y$ = \"not infected\". Bound the share of people who are protected (not infected with the vaccine, infected without), without assuming monotonicity. What does monotonicity add?\n", "\n", "
Show solution\n", "\n", "**Step 1.** $P(y_1) = 0.98$ and $P(y_0) = 0.90$; $P(y'_0) = 0.10$.\n", "\n", "**Step 2.** Lower bound: $\\max\\{0, 0.98 - 0.90\\} = 0.08$.\n", "\n", "**Step 3.** Upper bound: $\\min\\{0.98, 0.10\\} = 0.10$.\n", "\n", "**Step 4.** With monotonicity (the vaccine never causes infection), $PNS = 0.98 - 0.90 = 0.08$ exactly.\n", "\n", "**Answer:** between 8% and 10% of people are protected; at most 2% could be harmed (infected only because vaccinated), which is what widens the interval.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Effect of training on the trained\n", "\n", "Use Day 52's training data: half previously employed ($Z = 1$); training rates 30% ($Z = 1$) and 70% ($Z = 0$); employment $P(E = 1 \\mid T, Z)$: $Z = 1$: 0.60 untrained, 0.70 trained; $Z = 0$: 0.20 untrained, 0.35 trained. Compute $E[E_0 \\mid T = 1]$ and the ATT.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Composition of the trained: $P(Z = 1 \\mid T = 1) = \\dfrac{0.5(0.3)}{0.5} = 0.3$.\n", "\n", "**Step 2.** With $Z$ a back-door set, $E[E_0 \\mid T = 1] = \\sum_z P(E = 1 \\mid T = 0, z)P(z \\mid T = 1) = 0.3(0.60) + 0.7(0.20) = 0.32$.\n", "\n", "**Step 3.** $E[E \\mid T = 1] = 0.3(0.70) + 0.7(0.35) = 0.455$.\n", "\n", "**Step 4.** $ATT = 0.455 - 0.32 = 0.135$.\n", "\n", "**Answer:** training raised the employment of those trained by 13.5 points, slightly more than the ATE of 12.5 points, because the trained are mostly people for whom training helps more.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · A patient's counterfactual\n", "\n", "A patient refused a drug and died. Her family asks whether she would have survived with it. A trial shows survival of 70% with the drug and 60% without. What can be said?\n", "\n", "
Show solution\n", "\n", "**Step 1.** The question is rung 3: $P(\\text{survive}_1 \\mid X = 0, \\text{died})$, the probability of sufficiency.\n", "\n", "**Step 2.** Patients who refuse may differ from trial participants (refusal is not random), so even population-level quantities require care.\n", "\n", "**Step 3.** Treating her as exchangeable with trial patients and assuming monotonicity: $PS = \\dfrac{P(y \\mid x) - P(y \\mid x')}{1 - P(y \\mid x')} = \\dfrac{0.70 - 0.60}{0.40} = 0.25$.\n", "\n", "**Answer:** about a 25% chance she would have survived, under strong assumptions. Without monotonicity, only bounds are available; the honest answer is a range.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "# Exercise 3: abduction-action-prediction\n", "X, M, Y = 2, 1.5, 4\n", "uM, uY = M - .5 * X, Y - M - X\n", "M0 = .5 * 0 + uM; Y0 = M0 + 0 + uY\n", "print(\"M_0 =\", M0, \" Y_0 =\", Y0)\n", "\n", "# Exercise 11: counterfactual log wage\n", "resid = 2.6 - 1.5 - .08 * 12\n", "print(\"log W_16 =\", round(1.5 + .08 * 16 + resid, 3), \" wage gain\", f\"{np.exp(.32) - 1:.1%}\")\n", "\n", "# Exercises 4, 12, 13: necessity and sufficiency\n", "PN = lambda p1, p0: (p1 - p0) / p1\n", "print(\"PN drug:\", round(PN(.8, .5), 3), \" PN asbestos:\", round(PN(.02, .005), 3))\n", "p1, p0 = .98, .90\n", "print(\"PNS bounds:\", (round(max(0, p1 - p0), 3), round(min(p1, 1 - p0), 3)))\n", "\n", "# Exercise 10: two models, same experiments, different counterfactuals\n", "rng = np.random.default_rng(59); U = rng.random(1_000_000) < .5\n", "for name, f in [(\"A: Y = U\", lambda x: U), (\"B: Y = X xor U\", lambda x: np.logical_xor(x, U))]:\n", " Y1, Y0 = f(1), f(0)\n", " print(f\"{name:15s} P(Y=1|do1) {Y1.mean():.3f} P(Y=1|do0) {Y0.mean():.3f} PNS {(Y1 & ~Y0).mean():.3f}\")\n", "\n", "# Exercise 14: ATT\n", "pT = {1: .3, 0: .7}; pE = {(1, 0): .60, (1, 1): .70, (0, 0): .20, (0, 1): .35} # key (z, t)\n", "pz_t1 = {z: .5 * pT[z] / .5 for z in (0, 1)}\n", "EY0_T1 = sum(pE[z, 0] * pz_t1[z] for z in (0, 1)); EY_T1 = sum(pE[z, 1] * pz_t1[z] for z in (0, 1))\n", "print(f\"E[E_0|T=1] {EY0_T1:.3f} E[E|T=1] {EY_T1:.3f} ATT {EY_T1 - EY0_T1:.3f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- Rung 3 conditions counterfactuals on what actually happened; computing them takes abduction, action and prediction in an SCM.\n", "- In linear additive models every unit has the same effect; heterogeneity is what makes counterfactuals hard.\n", "- Probabilities of necessity and sufficiency answer attribution questions; they are identified under exogeneity and monotonicity and otherwise bounded." ] } ] }