{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 31: The Rubin Causal Model & Potential Outcomes\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "Units, treatments and potential outcomes; the science table; and why the assignment mechanism decides what the data can tell you.\n", "\n", "**Learning outcomes**\n", "\n", "- Write a science table and compute unit-level and average treatment effects.\n", "- Derive the observed-outcome identity Y = Y(0) + T·(Y(1) − Y(0)).\n", "- Prove that complete randomisation makes the difference in means unbiased for the finite-population ATE.\n", "- Distinguish well-defined from ill-defined treatments, and finite- from super-population estimands." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### Potential outcomes\n", "\n", "Phase 1 and 2 gave you the tools; Phase 3 gives them a causal meaning. The **Rubin causal model** (Neyman 1923, Rubin 1974) starts from three ingredients:\n", "\n", "- **Units** $i = 1, \\dots, n$: workers, households, patients, villages.\n", "- A **treatment** $T_i \\in \\{0, 1\\}$: training or not, a cash transfer or not.\n", "- **Potential outcomes** $Y_i(1)$ and $Y_i(0)$: what unit $i$'s outcome *would be* under each treatment, both defined before treatment is assigned.\n", "\n", "The **unit-level causal effect** is $\\tau_i = Y_i(1) - Y_i(0)$. Collected in one table, the potential outcomes and covariates of all units form the **science table**: everything there is to know about the causal question.\n", "\n", "### The observed outcome\n", "\n", "We only ever see the outcome under the treatment actually received:\n", "\n", "$$\n", "Y_i = T_iY_i(1) + (1 - T_i)Y_i(0) = Y_i(0) + T_i\\tau_i.\n", "$$\n", "\n", "For every unit, one potential outcome is observed and the other is **missing**: the *fundamental problem of causal inference* (Holland 1986). Causal inference is therefore a missing-data problem (Day 14).\n", "\n", "### Estimands\n", "\n", "For a fixed set of $n$ units, the **finite-population ATE** is\n", "\n", "$$\n", "\\tau_{fp} = \\frac1n\\sum_{i=1}^n\\big(Y_i(1) - Y_i(0)\\big) = \\bar Y(1) - \\bar Y(0).\n", "$$\n", "\n", "If the units are a random sample from a larger population, the **super-population ATE** is $E[Y(1) - Y(0)]$. Day 33 adds ATT, ATC and CATE.\n", "\n", "### The assignment mechanism\n", "\n", "The **assignment mechanism** is the rule that decides who gets treated: $P(T_1, \\dots, T_n \\mid \\text{science table})$. It is the crux of every design:\n", "\n", "- In a **randomised experiment** the researcher controls it and it does not depend on potential outcomes.\n", "- In an **observational study** people (or officials) choose, often based on what they expect to gain.\n", "\n", "With **complete randomisation**, exactly $n_1$ of $n$ units are treated and each of the $\\binom{n}{n_1}$ assignments is equally likely. Then the difference in means $\\hat\\tau = \\bar Y_1 - \\bar Y_0$ is unbiased for $\\tau_{fp}$ (Exercise 8).\n", "\n", "### What counts as a treatment\n", "\n", "\"No causation without manipulation\" (Holland): $Y(1)$ must be well defined. \"The effect of a job-training voucher\" is; \"the effect of being obese\" is not, because there are many ways to change weight (diet, exercise, surgery) with different consequences. Day 32 formalises this as SUTVA.\n", "\n", "## Part 1 · Warm-up\n", "\n", "Four unemployed workers' annual earnings (in \\$1,000) with and without a training programme:\n", "\n", "| Worker | $Y(0)$ | $Y(1)$ |\n", "|---|---|---|\n", "| A | 10 | 14 |\n", "| B | 12 | 12 |\n", "| C | 8 | 13 |\n", "| D | 15 | 15 |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Unit and average effects\n", "\n", "Compute each worker's effect and the finite-population ATE.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Unit effects $\\tau_i = Y_i(1) - Y_i(0)$: A $= 4$, B $= 0$, C $= 5$, D $= 0$.\n", "\n", "**Step 2.** ATE $= (4 + 0 + 5 + 0)/4 = 2.25$.\n", "\n", "**Answer:** training raises earnings by \\$2,250 on average, but only two of the four workers gain at all.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · What the data show\n", "\n", "Workers A and C are trained ($T = (1, 0, 1, 0)$). Write the observed outcomes and compute the difference in means.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Observed outcomes: A and C show $Y(1)$, B and D show $Y(0)$: $Y = (14, 12, 13, 15)$.\n", "\n", "**Step 2.** Treated mean $(14 + 13)/2 = 13.5$; control mean $(12 + 15)/2 = 13.5$.\n", "\n", "**Answer:** the difference is 0, although the ATE is 2.25. One assignment can give a misleading answer.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · The observed-outcome identity\n", "\n", "Check $Y = Y(0) + T\\tau$ for workers A and B.\n", "\n", "
Show solution\n", "\n", "**Step 1.** A: $10 + 1 \\times 4 = 14$ ✓ (observed $Y(1)$).\n", "\n", "**Step 2.** B: $12 + 0 \\times 0 = 12$ ✓ (observed $Y(0)$).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · The missing half\n", "\n", "For the assignment in Exercise 2, which entries of the science table are observed and which are missing?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Observed: $Y_A(1), Y_B(0), Y_C(1), Y_D(0)$.\n", "\n", "**Step 2.** Missing: $Y_A(0), Y_B(1), Y_C(0), Y_D(1)$.\n", "\n", "**Answer:** exactly half the table is missing, and every unit-level effect needs one missing value.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Counting assignments\n", "\n", "How many assignments treat exactly 2 of the 4 workers?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Choose 2 of 4: $\\binom42 = \\frac{4!}{2!\\,2!} = 6$.\n", "\n", "**Answer:** 6 equally likely assignments under complete randomisation.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · The observed-outcome identity\n", "\n", "Prove $T_iY_i(1) + (1 - T_i)Y_i(0) = Y_i(0) + T_i\\tau_i$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Expand the left side: $T_iY_i(1) + Y_i(0) - T_iY_i(0)$.\n", "\n", "**Step 2.** Group the terms with $T_i$: $Y_i(0) + T_i\\big(Y_i(1) - Y_i(0)\\big) = Y_i(0) + T_i\\tau_i$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · ATE as a difference of means\n", "\n", "Prove $\\tau_{fp} = \\bar Y(1) - \\bar Y(0)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\tau_{fp} = \\frac1n\\sum_i(Y_i(1) - Y_i(0))$.\n", "\n", "**Step 2.** Split the sum (linearity): $\\frac1n\\sum_iY_i(1) - \\frac1n\\sum_iY_i(0) = \\bar Y(1) - \\bar Y(0)$. $\\blacksquare$\n", "\n", "So we never need individual effects to learn the ATE, only the two means.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Complete randomisation is unbiased\n", "\n", "Under complete randomisation with $n_1$ treated, prove $E[\\bar Y_1 - \\bar Y_0] = \\tau_{fp}$, where the expectation is over assignments and the science table is fixed.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Write the treated mean with indicators: $\\bar Y_1 = \\frac{1}{n_1}\\sum_iT_iY_i(1)$ (for treated units the observed outcome is $Y_i(1)$).\n", "\n", "**Step 2.** Only $T_i$ is random, and by symmetry $E[T_i] = P(T_i = 1) = n_1/n$.\n", "\n", "**Step 3.** So $E[\\bar Y_1] = \\frac{1}{n_1}\\sum_i\\frac{n_1}{n}Y_i(1) = \\bar Y(1)$.\n", "\n", "**Step 4.** Likewise $\\bar Y_0 = \\frac{1}{n_0}\\sum_i(1 - T_i)Y_i(0)$ and $E[\\bar Y_0] = \\bar Y(0)$.\n", "\n", "**Step 5.** Linearity and Exercise 7: $E[\\bar Y_1 - \\bar Y_0] = \\bar Y(1) - \\bar Y(0) = \\tau_{fp}$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Unit effects are not identified\n", "\n", "Construct a second science table for workers A–D that produces exactly the same observed data as Exercise 2 but has a different ATE.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Keep the observed entries fixed: $Y_A(1) = 14$, $Y_B(0) = 12$, $Y_C(1) = 13$, $Y_D(0) = 15$.\n", "\n", "**Step 2.** Change the missing entries, e.g. $Y_A(0) = 14$, $Y_B(1) = 20$, $Y_C(0) = 13$, $Y_D(1) = 15$.\n", "\n", "**Step 3.** New effects: 0, 8, 0, 0, so ATE $= 2$ instead of 2.25.\n", "\n", "**Answer:** both tables fit the data perfectly. Without assumptions on the assignment mechanism, the data cannot distinguish them.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Post-treatment variables have potential values too\n", "\n", "A training programme may change whether a worker moves city, $M$. Explain why $M$ has potential values $M(1), M(0)$, and why comparing trained and untrained workers *who did not move* is not a causal comparison.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Moving happens after assignment and can be affected by it, so each worker has $M_i(1)$ and $M_i(0)$.\n", "\n", "**Step 2.** \"Trained non-movers\" are workers with $M_i(1) = 0$; \"untrained non-movers\" have $M_i(0) = 0$. These are generally different sets of people.\n", "\n", "**Answer:** the comparison mixes the effect of training with differences between two selected groups. Only pre-treatment covariates (fixed before assignment) can be conditioned on safely; this is the \"bad control\" problem of Day 53.\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Every possible experiment\n", "\n", "For the training science table, compute the difference in means for all 6 assignments with 2 trained workers, and verify that their average equals the ATE.\n", "\n", "
Show solution\n", "\n", "**Step 1.** For each pair treated, compare the treated workers' $Y(1)$ with the others' $Y(0)$:\n", "\n", "| Trained | Treated mean | Control mean | $\\hat\\tau$ |\n", "|---|---|---|---|\n", "| A, B | $(14 + 12)/2 = 13$ | $(8 + 15)/2 = 11.5$ | 1.5 |\n", "| A, C | 13.5 | 13.5 | 0 |\n", "| A, D | 14.5 | 10 | 4.5 |\n", "| B, C | 12.5 | 12.5 | 0 |\n", "| B, D | 13.5 | 9 | 4.5 |\n", "| C, D | 14 | 11 | 3 |\n", "\n", "**Step 2.** Average: $(1.5 + 0 + 4.5 + 0 + 4.5 + 3)/6 = 13.5/6 = 2.25$.\n", "\n", "**Answer:** exactly the ATE, as Exercise 8 promised, even though individual experiments range from 0 to 4.5. Unbiasedness is a property of the *design*, not of one realised experiment.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Workers who choose\n", "\n", "Now suppose workers know their gains and enrol only if they gain (A and C). Compute the difference in means and compare with the ATT.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Same observed data as Exercise 2: difference in means $= 0$.\n", "\n", "**Step 2.** ATT $=$ average effect among enrollees A and C $= (4 + 5)/2 = 4.5$.\n", "\n", "**Step 3.** Why the gap: enrollees would have earned less without training ($Y(0)$: 10 and 8) than non-enrollees (12 and 15), so the comparison is contaminated by selection bias $= \\bar Y_{A,C}(0) - \\bar Y_{B,D}(0) = 9 - 13.5 = -4.5$.\n", "\n", "**Answer:** naive 0 = ATT 4.5 + selection bias (−4.5). Self-selection can hide a large effect completely (Day 34).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Is the treatment well defined?\n", "\n", "A ministry asks: \"What is the effect of obesity on annual health-care costs?\" Explain why this is ill-posed in the potential-outcomes framework, and reformulate it as a well-defined question.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $Y_i(\\text{not obese})$ is ambiguous: losing weight through exercise, diet, medication or surgery would change costs differently.\n", "\n", "**Step 2.** So \"the\" counterfactual is not unique: SUTVA's \"no hidden versions\" fails (Day 32).\n", "\n", "**Step 3.** A well-defined version names a manipulable intervention, e.g. \"the effect on costs of offering a 12-month subsidised gym membership\" or \"of a sugar-sweetened-beverage tax\".\n", "\n", "**Answer:** causal questions need a specific intervention; the reformulation also matches what a policymaker can actually do.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Finite or super population?\n", "\n", "An evaluation studies all 120 municipalities of a province, randomising a governance reform. Another samples 2,000 households at random from a country and randomises a cash transfer. Which estimand is natural in each case?\n", "\n", "
Show solution\n", "\n", "**Step 1.** The 120 municipalities *are* the population of interest; uncertainty comes only from which ones were assigned the reform: the finite-population ATE is natural.\n", "\n", "**Step 2.** The 2,000 households stand in for millions; uncertainty comes from sampling *and* assignment: the super-population ATE is natural.\n", "\n", "**Answer:** finite-population for the municipalities, super-population for the households. Day 36 shows the two lead to slightly different variance formulas.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Who does a vaccine help?\n", "\n", "Six people are classified by their potential infection outcomes: 1 \"doomed\" ($Y(0) = Y(1) = 1$), 2 \"helped\" ($Y(0) = 1$, $Y(1) = 0$), 3 \"immune\" ($Y(0) = Y(1) = 0$). Compute the infection rate with and without the vaccine and the ATE. Could a trial reveal the number helped?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Without the vaccine, infected = doomed + helped = 3 of 6: $\\bar Y(0) = 0.5$.\n", "\n", "**Step 2.** With the vaccine, infected = doomed = 1 of 6: $\\bar Y(1) \\approx 0.167$.\n", "\n", "**Step 3.** ATE $= 0.167 - 0.5 \\approx -0.333$: the vaccine cuts the infection rate by a third of the population.\n", "\n", "**Step 4.** A trial identifies the two rates, hence the ATE. But a fourth type, \"hurt\" ($Y(0) = 0$, $Y(1) = 1$), would also be possible: 3 helped and 1 hurt give the same rates as 2 helped and 0 hurt.\n", "\n", "**Answer:** ATE = −1/3. The share helped equals −ATE only if nobody is hurt (a monotonicity assumption); the data alone cannot tell.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import itertools\n", "import numpy as np\n", "\n", "y0 = np.array([10, 12, 8, 15.]); y1 = np.array([14, 12, 13, 15.])\n", "print(\"unit effects:\", y1 - y0, \" ATE:\", (y1 - y0).mean())\n", "\n", "ests = []\n", "for treated in itertools.combinations(range(4), 2):\n", " T = np.zeros(4, bool); T[list(treated)] = True\n", " Y = np.where(T, y1, y0) # observed outcomes\n", " ests.append(Y[T].mean() - Y[~T].mean())\n", " print(\"treated\", [(\"ABCD\")[i] for i in treated], \"estimate\", ests[-1])\n", "print(\"average over all assignments:\", np.mean(ests), \"(= ATE)\")\n", "print(\"variance across assignments:\", round(np.var(ests), 4))\n", "\n", "# Self-selection (Exercise 12)\n", "T = (y1 - y0) > 0; Y = np.where(T, y1, y0)\n", "print(\"self-selected comparison:\", Y[T].mean() - Y[~T].mean(), \" ATT:\", (y1 - y0)[T].mean())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- A causal effect compares two potential outcomes of the same unit; only one is ever observed.\n", "- The assignment mechanism decides whether observed comparisons recover causal effects.\n", "- Complete randomisation makes the difference in means unbiased for the ATE, averaging over all possible assignments." ] } ] }