{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 67: Difference-in-Differences: the 2×2 Case\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "Compare how outcomes changed in a treated group with how they changed in an untreated group. Under parallel trends, the difference in changes is the effect of treatment on the treated. The estimand, its regression form, and the assumptions that make or break it.\n", "\n", "**Learning outcomes**\n", "\n", "- Compute the 2×2 difference-in-differences estimate from four means and from a regression with an interaction.\n", "- Prove that DiD identifies the ATT under parallel trends and no anticipation.\n", "- Show that parallel trends depend on the scale of the outcome (levels versus logs).\n", "- Apply DiD to minimum wages, health insurance expansions and training programmes, and spot failures such as the Ashenfelter dip." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### Set-up\n", "\n", "Two groups ($G = 1$ treated, $G = 0$ comparison) and two periods ($T = 0$ before, $T = 1$ after). Only the treated group in period 1 receives treatment. The target is the **ATT** in the post period:\n", "\n", "$$\n", "ATT = E[Y_{1}(1) - Y_{1}(0) \\mid G = 1],\n", "$$\n", "\n", "where $Y_t(d)$ is the potential outcome in period $t$. The problem: $E[Y_1(0) \\mid G = 1]$, what would have happened to the treated without treatment, is unobserved.\n", "\n", "### Two naive comparisons\n", "\n", "- **Before–after** in the treated group: $E[Y_1 - Y_0 \\mid G = 1]$, biased by anything else that changed over time.\n", "- **Treated–comparison** after: $E[Y_1 \\mid G = 1] - E[Y_1 \\mid G = 0]$, biased by permanent group differences.\n", "\n", "### Parallel trends\n", "\n", "**Assumption (parallel trends):** without treatment, the average outcome would have changed by the same amount in both groups:\n", "\n", "$$\n", "E[Y_1(0) - Y_0(0) \\mid G = 1] = E[Y_1(0) - Y_0(0) \\mid G = 0].\n", "$$\n", "\n", "**No anticipation:** $Y_0 = Y_0(0)$ for the treated (treatment does not affect pre-period outcomes).\n", "\n", "### The DiD estimand\n", "\n", "Under both assumptions,\n", "\n", "$$\n", "ATT = \\big(E[Y_1 \\mid G = 1] - E[Y_0 \\mid G = 1]\\big) - \\big(E[Y_1 \\mid G = 0] - E[Y_0 \\mid G = 0]\\big).\n", "$$\n", "\n", "The comparison group's change is the counterfactual change for the treated.\n", "\n", "### Regression form\n", "\n", "$$\n", "Y_{it} = \\alpha + \\gamma G_i + \\lambda T_t + \\delta\\,(G_i \\times T_t) + \\varepsilon_{it}.\n", "$$\n", "\n", "The four cell means are $\\alpha$, $\\alpha + \\lambda$, $\\alpha + \\gamma$, $\\alpha + \\gamma + \\lambda + \\delta$; so $\\hat\\delta$ is exactly the DiD of sample means. With panel data, regressing $\\Delta Y_i$ on $G_i$ gives the same number. Covariates can be added (for instance pre-treatment characteristics interacted with $T$), and standard errors should be clustered at the level at which treatment is assigned.\n", "\n", "### Scale matters\n", "\n", "Parallel trends in levels and in logs cannot both hold unless baseline levels are equal or there is no trend. The choice of scale is a substantive assumption, not a technicality.\n", "\n", "### Threats\n", "\n", "Different shocks to the two groups in the post period; selection into treatment based on transitory outcomes (the **Ashenfelter dip**: people join training after a temporary earnings drop, which would have recovered anyway); compositional changes in repeated cross-sections; spillovers to the comparison group.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Four means\n", "\n", "Treated: 40 before, 52 after. Comparison: 35 before, 41 after. Compute the DiD.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Treated change: $52 - 40 = 12$.\n", "\n", "**Step 2.** Comparison change: $41 - 35 = 6$.\n", "\n", "**Step 3.** DiD: $12 - 6 = 6$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Regression coefficients\n", "\n", "Using Exercise 1's means, give $\\alpha, \\gamma, \\lambda, \\delta$ in the regression form.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\alpha = 35$ (comparison, before).\n", "\n", "**Step 2.** $\\gamma = 40 - 35 = 5$ (group gap before).\n", "\n", "**Step 3.** $\\lambda = 41 - 35 = 6$ (comparison trend).\n", "\n", "**Step 4.** $\\delta = 52 - (35 + 5 + 6) = 6$, the DiD.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · The naive comparisons\n", "\n", "With Exercise 1's means, compute the before–after and post-period cross-sectional comparisons, and say what biases each.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Before–after: 12; includes the common trend of 6.\n", "\n", "**Step 2.** Cross-section after: $52 - 41 = 11$; includes the permanent gap of 5.\n", "\n", "**Answer:** both overstate the effect (6); DiD removes both biases under parallel trends.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · Levels or logs?\n", "\n", "Comparison: 10 → 12. Treated: 20 → 25. Compute DiD in levels and in logs. If the treated would have grown by the comparison group's percentage, what is the effect in levels?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Levels: $(25 - 20) - (12 - 10) = 3$.\n", "\n", "**Step 2.** Logs: $\\ln(25/20) - \\ln(12/10) = 0.223 - 0.182 = 0.041$.\n", "\n", "**Step 3.** Under proportional (log) parallel trends, the treated counterfactual is $20 \\times 1.2 = 24$, so the effect is $25 - 24 = 1$.\n", "\n", "**Answer:** 3 under level trends, 1 under proportional trends: the scale assumption triples the estimate.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Anticipation\n", "\n", "A tax cut announced a year ahead is evaluated with DiD using the announcement year as \"before\". Why might this be a problem?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Firms may change behaviour once the cut is announced (for example, delaying investment until it takes effect).\n", "\n", "**Step 2.** Then the \"before\" outcome is already affected: no anticipation fails.\n", "\n", "**Answer:** use a baseline before the announcement.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · DiD identifies the ATT\n", "\n", "Prove that under parallel trends and no anticipation the DiD estimand equals the ATT.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Observed outcomes: $E[Y_1 \\mid G = 1] = E[Y_1(1) \\mid G = 1]$; $E[Y_0 \\mid G = 1] = E[Y_0(0) \\mid G = 1]$ (no anticipation); comparison outcomes are untreated potential outcomes.\n", "\n", "**Step 2.** DiD $= E[Y_1(1) - Y_0(0) \\mid G = 1] - E[Y_1(0) - Y_0(0) \\mid G = 0]$.\n", "\n", "**Step 3.** By parallel trends, replace the second term by $E[Y_1(0) - Y_0(0) \\mid G = 1]$.\n", "\n", "**Step 4.** DiD $= E[Y_1(1) - Y_1(0) \\mid G = 1] = ATT$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · The interaction coefficient\n", "\n", "Prove that OLS of the saturated regression $Y = \\alpha + \\gamma G + \\lambda T + \\delta GT + \\varepsilon$ gives $\\hat\\delta = (\\bar Y_{11} - \\bar Y_{10}) - (\\bar Y_{01} - \\bar Y_{00})$, with $\\bar Y_{gt}$ the cell means.\n", "\n", "
Show solution\n", "\n", "**Step 1.** The regression has four parameters and four cells: it is saturated, so fitted values equal cell means.\n", "\n", "**Step 2.** Fitted values: $\\bar Y_{00} = \\hat\\alpha$, $\\bar Y_{01} = \\hat\\alpha + \\hat\\lambda$, $\\bar Y_{10} = \\hat\\alpha + \\hat\\gamma$, $\\bar Y_{11} = \\hat\\alpha + \\hat\\gamma + \\hat\\lambda + \\hat\\delta$.\n", "\n", "**Step 3.** Solve: $\\hat\\delta = \\bar Y_{11} - \\bar Y_{10} - \\bar Y_{01} + \\bar Y_{00}$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · First differences\n", "\n", "With a balanced panel, show that regressing $\\Delta Y_i = Y_{i1} - Y_{i0}$ on a constant and $G_i$ gives the DiD estimate.\n", "\n", "
Show solution\n", "\n", "**Step 1.** A regression on a constant and a binary variable gives the difference in group means of the outcome.\n", "\n", "**Step 2.** $\\hat\\beta_G = \\overline{\\Delta Y}_{G=1} - \\overline{\\Delta Y}_{G=0} = (\\bar Y_{11} - \\bar Y_{10}) - (\\bar Y_{01} - \\bar Y_{00})$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Bias from non-parallel trends\n", "\n", "Without parallel trends, show DiD $= ATT + \\Delta$, where $\\Delta = E[Y_1(0) - Y_0(0) \\mid G = 1] - E[Y_1(0) - Y_0(0) \\mid G = 0]$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** From Exercise 6, Step 2: DiD $= E[Y_1(1) - Y_0(0) \\mid G = 1] - E[Y_1(0) - Y_0(0) \\mid G = 0]$.\n", "\n", "**Step 2.** Add and subtract $E[Y_1(0) \\mid G = 1]$: DiD $= ATT + E[Y_1(0) - Y_0(0) \\mid G = 1] - E[Y_1(0) - Y_0(0) \\mid G = 0]$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Scale dependence\n", "\n", "Suppose untreated outcomes grow by the same proportion $g > 0$ in both groups and baseline means differ ($\\mu_1 \\ne \\mu_0$). Show that parallel trends in levels fails.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Untreated change for group $k$: $\\mu_k(1 + g) - \\mu_k = g\\mu_k$.\n", "\n", "**Step 2.** Difference between groups: $g(\\mu_1 - \\mu_0) \\ne 0$.\n", "\n", "**Answer:** level trends differ by $g(\\mu_1 - \\mu_0)$, which is exactly the level-DiD bias. $\\blacksquare$\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · The New Jersey minimum wage\n", "\n", "In Card and Krueger's study, average full-time-equivalent employment per fast-food restaurant was 20.44 before and 21.03 after New Jersey raised its minimum wage, and 23.33 and 21.17 in neighbouring eastern Pennsylvania. Compute the DiD and state the identifying assumption.\n", "\n", "
Show solution\n", "\n", "**Step 1.** New Jersey change: $21.03 - 20.44 = 0.59$.\n", "\n", "**Step 2.** Pennsylvania change: $21.17 - 23.33 = -2.16$.\n", "\n", "**Step 3.** DiD: $0.59 - (-2.16) = 2.75$.\n", "\n", "**Answer:** about +2.75 FTE workers per restaurant, no evidence of job loss. Identifying assumption: without the increase, New Jersey restaurants would have followed Pennsylvania's employment trend.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Medicaid expansion and mortality\n", "\n", "Adult mortality per 100,000 fell from 320 to 310 in states that expanded Medicaid and from 330 to 328 in states that did not. Estimate the effect and give one threat to parallel trends.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Expansion change: −10. Non-expansion change: −2.\n", "\n", "**Step 2.** DiD: $-10 - (-2) = -8$ deaths per 100,000.\n", "\n", "**Step 3.** Threat: expansion states may have been on a different health trajectory (different opioid epidemics, economic recoveries).\n", "\n", "**Answer:** −8 per 100,000, credible only if pre-expansion mortality trends were similar (Day 68 checks this).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · A pre-period placebo for bed nets\n", "\n", "Malaria cases per 1,000 children in districts that received a bed-net campaign in 2020: 2018: 150, 2019: 140, 2021: 100. In comparison districts: 2018: 160, 2019: 155, 2021: 140. Compute a placebo DiD for 2018–2019 and the main DiD for 2019–2021. Interpret.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Placebo: treated change $140 - 150 = -10$; comparison $155 - 160 = -5$; placebo DiD $= -5$.\n", "\n", "**Step 2.** Main: treated $100 - 140 = -40$; comparison $140 - 155 = -15$; DiD $= -25$.\n", "\n", "**Step 3.** The placebo shows the treated districts were already improving 5 cases faster per year.\n", "\n", "**Answer:** −25 per 1,000, but with a pre-trend; if the extra −5 per year continued over two years, a trend-adjusted estimate would be about $-25 + 10 = -15$ (Day 68 formalises this).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Inference with two states\n", "\n", "The New Jersey study compares restaurants in two states. Why are conventional standard errors (treating restaurants as independent) too optimistic, and what can be done?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Any state-specific shock in the post period hits all restaurants in that state together: errors are correlated within state.\n", "\n", "**Step 2.** With one treated and one comparison cluster, cluster-robust SEs cannot be computed reliably.\n", "\n", "**Answer:** use more comparison units (many states or counties), randomisation inference across placebo states, or synthetic control methods (Day 85); be explicit that with two clusters the uncertainty is understated.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · The Ashenfelter dip\n", "\n", "Trainees' earnings were \\$20,000 two years before training, \\$15,000 the year before (after job losses) and \\$19,000 the year after. Comparison workers earned \\$20,000, \\$19,800 and \\$20,000. Compute DiD using the year before as baseline, then using two years before. Which is more credible?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Baseline = year before: treated $+4{,}000$; comparison $+200$; DiD $= 3{,}800$.\n", "\n", "**Step 2.** Baseline = two years before: treated $-1{,}000$; comparison $0$; DiD $= -1{,}000$.\n", "\n", "**Step 3.** The dip just before training is transitory: trainees enrolled because their earnings had temporarily fallen, and much of the rebound would have happened anyway.\n", "\n", "**Answer:** the year-before baseline overstates the effect; analyses should use earlier baselines, model the dip, or compare with workers who had similar dips.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "did = lambda t0, t1, c0, c1: (t1 - t0) - (c1 - c0)\n", "print(\"Ex 1:\", did(40, 52, 35, 41), \" NJ:\", round(did(20.44, 21.03, 23.33, 21.17), 2), \" Medicaid:\", did(320, 310, 330, 328))\n", "print(\"logs vs levels:\", did(20, 25, 10, 12), round(np.log(25 / 20) - np.log(12 / 10), 4))\n", "print(\"bed nets placebo:\", did(150, 140, 160, 155), \" main:\", did(140, 100, 155, 140))\n", "print(\"Ashenfelter:\", did(15_000, 19_000, 19_800, 20_000), did(20_000, 19_000, 20_000, 20_000))\n", "\n", "# Regression form with simulated restaurants: the interaction equals the DiD of means\n", "rng = np.random.default_rng(67)\n", "n = 400; G = np.repeat([0, 1], n // 2)\n", "y0 = 23 - 3 * G + rng.normal(0, 8, n) # before\n", "y1 = y0 - 2 + 2.75 * G + rng.normal(0, 4, n) # after: common trend -2, effect 2.75\n", "Y = np.r_[y0, y1]; T = np.r_[np.zeros(n), np.ones(n)]; GG = np.r_[G, G]\n", "X = np.column_stack([np.ones(2 * n), GG, T, GG * T])\n", "b = np.linalg.lstsq(X, Y, rcond=None)[0]\n", "means = did(y0[G == 1].mean(), y1[G == 1].mean(), y0[G == 0].mean(), y1[G == 0].mean())\n", "fd = np.polyfit(G, y1 - y0, 1)[0]\n", "print(f\"interaction {b[3]:.3f} DiD of means {means:.3f} first-difference regression {fd:.3f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- DiD = (change for the treated) − (change for the comparison group); it identifies the ATT under parallel trends and no anticipation.\n", "- The regression interaction, the first-difference regression and the four-means formula are the same estimate.\n", "- Parallel trends is scale-dependent and fails with transitory pre-treatment shocks such as the Ashenfelter dip." ] } ] }