{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 71: Heterogeneity-Robust DiD (Callaway–Sant'Anna, Sun–Abraham)\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "Estimate a separate DiD for every adoption cohort and period using only clean controls, then average them with weights you choose. Group-time ATTs, their event-time and overall aggregations, the interaction-weighted event study, and how these fix the problems of staggered TWFE.\n", "\n", "**Learning outcomes**\n", "\n", "- Define and compute group-time average treatment effects ATT(g, t) with never-treated or not-yet-treated controls.\n", "- Aggregate ATT(g, t) into event-study, cohort and overall summaries with explicit weights.\n", "- Prove identification of ATT(g, t) under parallel trends and no anticipation, and the role of anticipation for not-yet-treated controls.\n", "- Explain why TWFE event studies are contaminated and how the interaction-weighted estimator avoids it." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### Cohorts and group-time effects\n", "\n", "Let $G_i$ be the period in which unit $i$ first receives treatment (its **cohort**), with $G_i = \\infty$ for never-treated units. The building block is the **group-time ATT**:\n", "\n", "$$\n", "ATT(g, t) = E[Y_t(g) - Y_t(\\infty) \\mid G = g],\n", "$$\n", "\n", "the effect in period $t$ for units that adopted at $g$.\n", "\n", "### Identification (Callaway and Sant'Anna)\n", "\n", "Under parallel trends and no anticipation, with never-treated controls,\n", "\n", "$$\n", "ATT(g, t) = E[Y_t - Y_{g-1} \\mid G = g] - E[Y_t - Y_{g-1} \\mid G = \\infty].\n", "$$\n", "\n", "Each $ATT(g, t)$ is a clean 2×2 DiD: cohort $g$ versus never-treated units, between the last pre-period $g - 1$ and period $t$. With **not-yet-treated** controls, replace the second term by the change among units with $G > t$ (untreated through period $t$). Already-treated units are never used as controls.\n", "\n", "Pre-treatment $ATT(g, t)$ with $t < g$ (computed relative to the preceding period, or to $g - 1$) serve as placebo tests.\n", "\n", "### Aggregation\n", "\n", "Choose weights for the question asked:\n", "\n", "- **Event study:** $\\theta(e) = \\sum_g w_g\\,ATT(g, g + e)$, with $w_g$ the share of cohort $g$ among cohorts observed $e$ periods after adoption.\n", "- **Overall:** average of all post-treatment $ATT(g, t)$ weighted by cohort size (each treated unit-period equally), or average of the event-study $\\theta(e)$.\n", "- **By cohort or calendar time** as needed.\n", "\n", "Weights are non-negative and transparent, unlike TWFE's.\n", "\n", "### Sun and Abraham's interaction-weighted estimator\n", "\n", "A TWFE event-study regression with relative-time dummies pools cohorts; Sun and Abraham (2021) show each coefficient can be contaminated by effects at **other** relative times. Their fix: interact relative-time dummies with cohort dummies (using never- or last-treated units as controls), estimate cohort-specific effects $CATT(g, e)$, then average them with cohort shares. This equals the Callaway–Sant'Anna event study with never-treated controls in the basic case.\n", "\n", "### Other robust estimators\n", "\n", "The **imputation** estimator (Borusyak, Jaravel and Spiess) fits unit and time effects on untreated observations only, imputes $Y(0)$ for treated observations, and averages the differences. Covariates enter through conditional parallel trends, with outcome-regression, weighting or doubly robust versions.\n", "\n", "### Practical choices\n", "\n", "Never-treated versus not-yet-treated controls (more data versus fewer assumptions about anticipation); balanced event-time samples (so the event study does not change composition); clustered inference across units.\n", "\n", "## Part 1 · Warm-up\n", "\n", "Exercises 1–5 use these mean outcomes (equal cohort sizes):\n", "\n", "| Cohort | t = 1 | t = 2 | t = 3 | t = 4 |\n", "|---|---|---|---|---|\n", "| $g = 2$ | 10 | 13 | 15 | 17 |\n", "| $g = 3$ | 12 | 13 | 16 | 18 |\n", "| never | 8 | 9 | 10 | 11 |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Group-time ATTs\n", "\n", "Compute $ATT(2, 2), ATT(2, 3), ATT(2, 4), ATT(3, 3), ATT(3, 4)$ with never-treated controls.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Cohort 2, base period 1: $ATT(2, 2) = (13 - 10) - (9 - 8) = 2$; $ATT(2, 3) = (15 - 10) - (10 - 8) = 3$; $ATT(2, 4) = (17 - 10) - (11 - 8) = 4$.\n", "\n", "**Step 2.** Cohort 3, base period 2: $ATT(3, 3) = (16 - 13) - (10 - 9) = 2$; $ATT(3, 4) = (18 - 13) - (11 - 9) = 3$.\n", "\n", "**Answer:** effects of 2, 3, 4 at event times 0, 1, 2 in both cohorts.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Event-study aggregation\n", "\n", "Compute $\\theta(0)$, $\\theta(1)$, $\\theta(2)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\theta(0) = (ATT(2, 2) + ATT(3, 3))/2 = 2$.\n", "\n", "**Step 2.** $\\theta(1) = (ATT(2, 3) + ATT(3, 4))/2 = 3$.\n", "\n", "**Step 3.** $\\theta(2) = ATT(2, 4) = 4$ (only cohort 2 is observed two periods after adoption).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · An overall ATT\n", "\n", "Average all five post-treatment group-time effects with equal weight per treated cohort-period.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $(2 + 3 + 4 + 2 + 3)/5 = 14/5 = 2.8$.\n", "\n", "**Answer:** 2.8. (TWFE on these data gives 2.3, the code shows.)\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · A placebo\n", "\n", "Compute the pre-treatment placebo for cohort 3 between periods 1 and 2.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $(13 - 12) - (9 - 8) = 0$.\n", "\n", "**Answer:** zero: cohort 3 and the never-treated moved in parallel before adoption.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Not-yet-treated controls\n", "\n", "Recompute $ATT(2, 2)$ using as controls the never-treated and cohort 3 (not yet treated at $t = 2$), with equal weight.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Control change 1 → 2: never $+1$, cohort 3 $+1$; average $+1$.\n", "\n", "**Step 2.** $ATT(2, 2) = 3 - 1 = 2$.\n", "\n", "**Answer:** the same here; not-yet-treated controls add data when there are few never-treated units.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · Identifying ATT(g, t)\n", "\n", "Assume (i) no anticipation: $Y_s = Y_s(\\infty)$ for cohort $g$ when $s < g$; (ii) parallel trends: $E[Y_t(\\infty) - Y_{g-1}(\\infty) \\mid G = g] = E[Y_t(\\infty) - Y_{g-1}(\\infty) \\mid G = \\infty]$. Prove the identification formula.\n", "\n", "
Show solution\n", "\n", "**Step 1.** For cohort $g$ at $t \\ge g$: $Y_t = Y_t(g)$; at $g - 1$: $Y_{g-1} = Y_{g-1}(\\infty)$ by (i).\n", "\n", "**Step 2.** $E[Y_t - Y_{g-1} \\mid G = g] = ATT(g, t) + E[Y_t(\\infty) - Y_{g-1}(\\infty) \\mid G = g]$.\n", "\n", "**Step 3.** For never-treated units $Y = Y(\\infty)$, so by (ii) the second term equals $E[Y_t - Y_{g-1} \\mid G = \\infty]$.\n", "\n", "**Step 4.** Subtract. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · Not-yet-treated controls\n", "\n", "Show the formula remains valid with controls $\\{G > t\\}$ if parallel trends holds for them and none of them anticipates treatment before adopting.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Units with $G > t$ are untreated in periods $g - 1$ and $t$; with no anticipation, their observed outcomes are $Y(\\infty)$ in both periods.\n", "\n", "**Step 2.** Parallel trends with them gives the same argument as Exercise 6, Step 3. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Anticipation contaminates not-yet-treated controls\n", "\n", "Suppose cohort 3 changes behaviour in period 2 (one period early), raising its outcome by $a$. Show that $ATT(2, 2)$ estimated with never-treated and cohort-3 controls (equal weights) is biased by $-a/2$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Cohort 3's observed change from 1 to 2 is its untreated trend plus $a$.\n", "\n", "**Step 2.** The average control change is the untreated trend plus $a/2$.\n", "\n", "**Step 3.** The estimate is $ATT(2, 2) - a/2$. $\\blacksquare$ With never-treated controls only, there is no bias.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Aggregations are convex combinations\n", "\n", "Show that $\\theta(e) = \\sum_g w_g ATT(g, g + e)$ with $w_g = P(G = g \\mid G + e \\le T, G \\le T)$ has non-negative weights summing to 1.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Conditional probabilities over the cohorts observed at event time $e$ are non-negative.\n", "\n", "**Step 2.** They sum to 1 over those cohorts. $\\blacksquare$ Hence $\\theta(e)$ lies between the smallest and largest cohort effects at event time $e$, unlike TWFE (Day 70, Exercise 9).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Why TWFE event studies mislead\n", "\n", "In a staggered design, the TWFE event-study coefficient for relative time $e$ uses already-treated cohorts as part of the comparison for other cohorts. Using Day 70's Exercise 7, explain why a pre-period coefficient can be non-zero even when parallel trends holds.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Relative-time dummies are estimated jointly with unit and time effects; the time effects are partly identified from already-treated cohorts.\n", "\n", "**Step 2.** If those cohorts' effects evolve, the time effects absorb part of their dynamics.\n", "\n", "**Step 3.** Every relative-time coefficient (including leads) is then measured against these contaminated time effects, so treatment dynamics leak into the leads.\n", "\n", "**Answer:** spurious pre-trends (or spurious flat pre-trends) can arise from treatment effect heterogeneity; cohort-specific estimates avoid this. $\\blacksquare$\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Staggered state paid sick-leave mandates\n", "\n", "The table above records the share of workers with any sick leave used (%) in states adopting a mandate in period 2, in period 3, and never. Summarise the effects for a policy brief, and say which control group you would prefer.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Group-time effects: 2, 3, 4 points at event times 0, 1, 2 (Exercise 1).\n", "\n", "**Step 2.** Event study: 2 on impact, rising to 3 after a year and 4 after two (cohort 2 only).\n", "\n", "**Step 3.** Control choice: never-treated states, unless they are few or differ systematically; not-yet-treated states add precision if mandates were not anticipated.\n", "\n", "**Answer:** mandates raise sick-leave use by about 2 points immediately and 4 points after two years; overall ATT 2.8.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · TWFE versus group-time ATTs\n", "\n", "On the same data, TWFE gives 2.3 while the cell-weighted ATT is 2.8. Explain the gap.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Effects grow with time since adoption.\n", "\n", "**Step 2.** TWFE includes a comparison of cohort 3 with the already-treated cohort 2, whose effect rises from 2 to 3 to 4 during that window; the rise is subtracted.\n", "\n", "**Answer:** TWFE understates the effect by about 0.5 points; the robust estimator uses only never-treated or not-yet-treated controls.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Medicaid expansions in 2014, 2015, 2016 and never\n", "\n", "Outline the analysis of the effect of Medicaid expansion on uninsured rates with Callaway–Sant'Anna.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Cohorts: 2014, 2015, 2016 expanders; never-expanders as controls (or not-yet-expanded states for early years).\n", "\n", "**Step 2.** Estimate $ATT(g, t)$ for every cohort and year, with pre-period placebos.\n", "\n", "**Step 3.** Aggregate into an event study (balanced over, say, event times −3 to +3) and an overall ATT.\n", "\n", "**Step 4.** Add baseline covariates (pre-2014 uninsured rate, income) through conditional parallel trends with a doubly robust estimator; cluster by state.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Composition changes in the event study\n", "\n", "In Exercise 2, $\\theta(2)$ uses only cohort 2. Why can this mislead, and how can it be fixed?\n", "\n", "
Show solution\n", "\n", "**Step 1.** If cohorts differ in their effects, $\\theta(e)$ can change across $e$ because different cohorts enter, not because effects evolve.\n", "\n", "**Step 2.** Fix: restrict to a balanced event window where all cohorts used are observed at every $e$ (here $e = 0, 1$), or show cohort-specific profiles.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Interaction-weighted event study\n", "\n", "Explain to a colleague how to run Sun and Abraham's estimator with standard regression software.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Keep never-treated (or last-treated) units as the control cohort.\n", "\n", "**Step 2.** Regress the outcome on unit and time fixed effects plus interactions of each cohort with each relative-time dummy (excluding $e = -1$).\n", "\n", "**Step 3.** Average the cohort-specific coefficients at each $e$ using each cohort's share among units observed at $e$, and compute standard errors for these averages (delta method or bootstrap, clustered by unit).\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "Y = {2: np.array([10, 13, 15, 17.]), 3: np.array([12, 13, 16, 18.]), np.inf: np.array([8, 9, 10, 11.])}\n", "T = 4\n", "att = {}\n", "for g in (2, 3):\n", " for t in range(g, T + 1):\n", " att[g, t] = (Y[g][t - 1] - Y[g][g - 2]) - (Y[np.inf][t - 1] - Y[np.inf][g - 2])\n", "print(\"ATT(g,t):\", {k: float(v) for k, v in att.items()})\n", "theta = {e: np.mean([att[g, g + e] for g in (2, 3) if g + e <= T]) for e in range(0, 3)}\n", "print(\"event study:\", {e: float(v) for e, v in theta.items()}, \" overall:\", float(np.mean(list(att.values()))))\n", "print(\"placebo cohort 3 (1->2):\", (Y[3][1] - Y[3][0]) - (Y[np.inf][1] - Y[np.inf][0]))\n", "\n", "# TWFE on the same cohort-level panel\n", "M = np.array([Y[2], Y[3], Y[np.inf]]); D = np.array([[0, 1, 1, 1], [0, 0, 1, 1], [0, 0, 0, 0.]])\n", "dm = lambda A: A - A.mean(1, keepdims=True) - A.mean(0, keepdims=True) + A.mean()\n", "print(\"TWFE:\", round((dm(D) * M).sum() / (dm(D) ** 2).sum(), 3))\n", "\n", "# Unit-level simulation: many units per cohort, noise, the same dynamic effects\n", "rng = np.random.default_rng(71); n = 300\n", "cohorts = np.repeat([2, 3, 99], n); t = np.arange(1, T + 1)\n", "eff = lambda e: np.where(e >= 0, 2 + e, 0)\n", "Yu = rng.normal(0, 2, 3 * n)[:, None] + t[None, :] + eff(t[None, :] - cohorts[:, None]) + rng.normal(0, 1, (3 * n, T))\n", "def att_gt(g, tt):\n", " tr, nv = cohorts == g, cohorts == 99\n", " return (Yu[tr, tt - 1] - Yu[tr, g - 2]).mean() - (Yu[nv, tt - 1] - Yu[nv, g - 2]).mean()\n", "print(\"simulated ATT(g,t):\", {(g, tt): round(float(att_gt(g, tt)), 2) for g in (2, 3) for tt in range(g, T + 1)})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- $ATT(g, t)$ compares each adoption cohort with never-treated or not-yet-treated units relative to its own pre-period: no forbidden comparisons.\n", "- Aggregate with explicit, non-negative weights into event-study, cohort or overall effects; keep event windows balanced.\n", "- Sun–Abraham's interaction-weighted event study and imputation estimators solve the same contamination problem in regression form." ] } ] }