{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 72: Triple Differences & Changes-in-Changes\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "Two extensions of DiD. Triple differences add a within-state comparison group that is not targeted by the policy, so state-specific shocks cancel. Changes-in-changes drops additivity altogether and maps ranks across distributions, giving effects on whole outcome distributions that do not depend on the outcome's scale.\n", "\n", "**Learning outcomes**\n", "\n", "- Compute a triple-difference estimate and state the single parallel-trends assumption it needs.\n", "- Write the fully interacted DDD regression.\n", "- Derive the changes-in-changes counterfactual distribution and compute it by rank matching.\n", "- Choose between DiD, DDD and CiC in labour and health applications." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### Triple differences\n", "\n", "DiD fails if the treated state has its own shock in the post period. If the policy targets a **subgroup** (married women of child-bearing age; low-income adults), we can use an untargeted subgroup in the same states to absorb state shocks:\n", "\n", "$$\n", "DDD = \\underbrace{\\big(\\Delta\\bar Y_{T, A} - \\Delta\\bar Y_{C, A}\\big)}_{\\text{DiD, affected group}} - \\underbrace{\\big(\\Delta\\bar Y_{T, B} - \\Delta\\bar Y_{C, B}\\big)}_{\\text{DiD, unaffected group}},\n", "$$\n", "\n", "where $T, C$ index treated and control states, $A, B$ the affected and unaffected subgroups, and $\\Delta$ the before–after change.\n", "\n", "**Assumption:** the bias of the DiD (non-parallel trends) is the same for the two subgroups. DDD does not need two separate parallel-trends assumptions; it needs only that the *difference* between subgroups would have evolved in parallel across states (Olden and Møen).\n", "\n", "**Regression:** a saturated regression with state-group, period and subgroup indicators and all their interactions; the coefficient on the triple interaction $T \\times \\text{Post} \\times A$ is the DDD.\n", "\n", "### Changes-in-changes (Athey and Imbens)\n", "\n", "DiD assumes additivity: the counterfactual shift is the same in levels for everyone. CiC instead assumes\n", "\n", "- untreated outcomes are $Y(0) = h(U, t)$ with $h$ strictly increasing in an unobserved \"rank\" $U$;\n", "- the distribution of $U$ within each group does not change over time (it may differ between groups).\n", "\n", "Then a treated unit at rank $q$ in its group's pre-period distribution would have the outcome of the control-group rank that the same $U$ maps to. The counterfactual distribution of the treated in the post period is\n", "\n", "$$\n", "F_{Y(0), 11}(y) = F_{10}\\Big(F_{00}^{-1}\\big(F_{01}(y)\\big)\\Big),\n", "$$\n", "\n", "where $F_{gt}$ is the observed distribution in group $g$, period $t$. Equivalently, the counterfactual for a treated pre-period outcome $y$ is $F_{01}^{-1}(F_{00}(y))$: find $y$'s rank among control units before, then the control outcome at that rank after.\n", "\n", "### Why CiC\n", "\n", "CiC gives **quantile treatment effects** (effects at the median, at the 10th percentile), not just a mean, and is invariant to monotone transformations of the outcome: levels and logs give the same answer, resolving Day 67's scale problem. It requires continuous outcomes (or bounds for discrete ones) and support overlap.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · A triple difference\n", "\n", "Changes in log wages (%), before and after a state mandated maternity benefits: treated state, married women 20–40: −5; treated state, others: −1; control states, married women 20–40: +1; control states, others: +2. Compute the DiD for each group and the DDD.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Affected group DiD: $-5 - 1 = -6$.\n", "\n", "**Step 2.** Unaffected group DiD: $-1 - 2 = -3$.\n", "\n", "**Step 3.** DDD: $-6 - (-3) = -3$.\n", "\n", "**Answer:** −3%: the mandate's cost was partly shifted to targeted workers' wages. The DiD for the affected group alone (−6) includes a state-wide shock of −3.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · The regression coefficient\n", "\n", "In the saturated DDD regression, which coefficient is the DDD, and what do the two-way interactions absorb?\n", "\n", "
Show solution\n", "\n", "**Step 1.** The coefficient on $\\text{Treated state} \\times \\text{Post} \\times \\text{Affected}$.\n", "\n", "**Step 2.** $\\text{Treated state} \\times \\text{Post}$ absorbs state-wide shocks; $\\text{Affected} \\times \\text{Post}$ absorbs nationwide trends for the targeted group; $\\text{Treated state} \\times \\text{Affected}$ absorbs permanent differences.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · What DDD buys\n", "\n", "Give an example of a shock DiD cannot handle but DDD can, and one DDD cannot handle.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Handled: a recession hitting the treated state at the same time as the mandate, affecting all workers in the state equally.\n", "\n", "**Step 2.** Not handled: a second policy in the treated state at the same time that also targets married women (for example a child-care subsidy).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · One CiC counterfactual\n", "\n", "Control-group outcomes before: 5, 8, 12, 15; after: 6, 10, 15, 20. A treated unit had 12 before. Using ranks, what is its counterfactual post-period outcome? What does DiD's additive shift imply?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Rank of 12 among controls before: third of four ($F_{00}(12) = 0.75$).\n", "\n", "**Step 2.** Control outcome at the same rank after: 15.\n", "\n", "**Step 3.** CiC counterfactual: 15. DiD shift: control mean rises from 10 to 12.75 (+2.75), so DiD predicts $12 + 2.75 = 14.75$.\n", "\n", "**Answer:** CiC lets the change depend on the rank: higher-ranked units gained more.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Logs or levels?\n", "\n", "Why does CiC give the same estimated effects (after transformation) whether the outcome is wages or log wages?\n", "\n", "
Show solution\n", "\n", "**Step 1.** CiC uses only ranks and quantiles; a strictly increasing transformation preserves ranks.\n", "\n", "**Answer:** the counterfactual of $\\log Y$ is the log of the counterfactual of $Y$ (Exercise 9).\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · DDD removes common bias\n", "\n", "Let each subgroup DiD equal $ATT_s + b_s$, where $b_s$ is its parallel-trends bias, with $ATT_B = 0$ for the unaffected subgroup. Show $DDD = ATT_A$ if $b_A = b_B$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $DDD = (ATT_A + b_A) - (0 + b_B)$.\n", "\n", "**Step 2.** With $b_A = b_B$: $DDD = ATT_A$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · The triple interaction\n", "\n", "Show that in the saturated regression with all main effects and interactions of $S$ (treated state), $P$ (post) and $A$ (affected), the coefficient on $S \\times P \\times A$ is the DDD of cell means.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Eight cells and eight parameters: fitted values equal cell means.\n", "\n", "**Step 2.** Within $A = a$, the coefficients on $S \\times P$ plus ($a$ times) the triple interaction give the DiD for subgroup $a$.\n", "\n", "**Step 3.** Subtracting the $A = 0$ DiD from the $A = 1$ DiD leaves the triple-interaction coefficient. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · The CiC counterfactual\n", "\n", "Assume $Y(0) = h(U, t)$, $h$ strictly increasing in $U$, and $U \\mid G = g$ has the same distribution in both periods. Prove $F_{Y(0), 11}(y) = F_{10}(F_{00}^{-1}(F_{01}(y)))$ (continuous distributions).\n", "\n", "
Show solution\n", "\n", "**Step 1.** Controls: $F_{0t}(y) = P(h(U, t) \\le y \\mid G = 0) = F_{U \\mid 0}(h^{-1}(y, t))$.\n", "\n", "**Step 2.** So $h^{-1}(y, 1) = F_{U \\mid 0}^{-1}(F_{01}(y))$ and $h(u, 0) = F_{00}^{-1}(F_{U \\mid 0}(u))$; hence $h(h^{-1}(y, 1), 0) = F_{00}^{-1}(F_{01}(y))$.\n", "\n", "**Step 3.** Treated in period 1 without treatment: $F_{Y(0), 11}(y) = P(U \\le h^{-1}(y, 1) \\mid G = 1)$.\n", "\n", "**Step 4.** Since $U \\mid G = 1$ is stable, this equals $P(h(U, 0) \\le h(h^{-1}(y, 1), 0) \\mid G = 1) = F_{10}(F_{00}^{-1}(F_{01}(y)))$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Invariance to monotone transformations\n", "\n", "Let $g$ be strictly increasing and apply CiC to $g(Y)$. Show the counterfactual for a treated pre-period value $y$ becomes $g\\big(F_{01}^{-1}(F_{00}(y))\\big)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** The distribution of $g(Y)$ satisfies $F^g(g(y)) = F(y)$ and $(F^g)^{-1}(q) = g(F^{-1}(q))$.\n", "\n", "**Step 2.** Counterfactual for $g(y)$: $(F^g_{01})^{-1}(F^g_{00}(g(y))) = g(F_{01}^{-1}(F_{00}(y)))$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · When CiC equals DiD\n", "\n", "Suppose $h(u, t) = u + \\delta_t$ (an additive time shift). Show the CiC counterfactual mean equals the DiD counterfactual.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Controls' distribution shifts by $\\delta_1 - \\delta_0$: $F_{01}^{-1}(F_{00}(y)) = y + \\delta_1 - \\delta_0$.\n", "\n", "**Step 2.** Each treated unit's counterfactual is its pre-period outcome plus $\\delta_1 - \\delta_0$; averaging gives $\\bar Y_{10} + (\\bar Y_{01} - \\bar Y_{00})$.\n", "\n", "**Step 3.** That is the DiD counterfactual. $\\blacksquare$\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Mandated maternity benefits\n", "\n", "Exercise 1's numbers resemble Gruber's study of state maternity mandates. Interpret the DDD of −3% for workers and employers.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Married women of child-bearing age, the group whose expected health costs rose, saw relative wage declines of 3%.\n", "\n", "**Step 2.** Unaffected workers in the same states absorb the state-wide shock (−3% in the DiD for others).\n", "\n", "**Answer:** employers shifted most of the mandate's cost onto targeted workers' wages, with little employment change: incidence on workers.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Medicaid expansion by income\n", "\n", "Change in uninsured rate (points): expansion states, low-income adults −9; high-income adults −2. Non-expansion states: low-income −4; high-income −1. Compute the DDD and give its assumption.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Low-income DiD: $-9 - (-4) = -5$.\n", "\n", "**Step 2.** High-income DiD: $-2 - (-1) = -1$.\n", "\n", "**Step 3.** DDD: $-5 - (-1) = -4$ points.\n", "\n", "**Answer:** expansion cut low-income uninsurance by 4 points; assumption: absent expansion, the low–high income gap in uninsurance would have evolved the same way in both sets of states. (If high-income adults also benefited, through spillovers, the DDD understates the effect.)\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Distributional effects of a training programme\n", "\n", "In the simulation below, training raises earnings by more for workers with high latent earnings capacity. Compare the DiD estimate with CiC quantile effects at the 10th, 50th and 90th percentiles.\n", "\n", "
Show solution\n", "\n", "**Step 1.** DiD gives one number (the average effect).\n", "\n", "**Step 2.** CiC maps each treated unit's pre-period rank through the control group's distributions to build the counterfactual distribution, then compares quantiles.\n", "\n", "**Step 3.** The code shows quantile effects rising from slightly negative at the 10th percentile to large at the 90th, close to the simulated truth.\n", "\n", "**Step 4.** DiD (about 2.8) also overstates the mean effect (about 1.8), because untreated earnings grow multiplicatively, so additive parallel trends fail; CiC's mean effect (1.7) is close.\n", "\n", "**Answer:** CiC reveals that the programme helps those with high earnings capacity and may slightly hurt those at the bottom, information a mean DiD hides.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · A threat to DDD\n", "\n", "In Exercise 12, expansion states also launched an outreach campaign for low-income families in the same year. What happens to the DDD?\n", "\n", "
Show solution\n", "\n", "**Step 1.** The campaign affects the same state × group × period cell as the expansion.\n", "\n", "**Answer:** DDD attributes the campaign's effect to the expansion; it cannot separate policies that coincide in all three dimensions.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Which design?\n", "\n", "Choose DiD, DDD or CiC: (a) a hospital-quality programme in one region, with a coinciding regional recession that affects all hospitals; patients' conditions not targeted are also observed. (b) A minimum-wage increase whose effect on the whole wage distribution is of interest. (c) A national school-meals expansion compared with a neighbouring country, with no subgroup structure.\n", "\n", "
Show solution\n", "\n", "**Step 1.** (a) DDD: compare targeted with untargeted conditions within region to remove the recession.\n", "\n", "**Step 2.** (b) CiC (or distributional DiD) for quantile effects.\n", "\n", "**Step 3.** (c) DiD, after checking pre-trends and scale.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "# Exercises 1 and 12\n", "ddd = lambda tA, cA, tB, cB: (tA - cA) - (tB - cB)\n", "print(\"maternity DDD:\", ddd(-5, 1, -1, 2), \" Medicaid DDD:\", ddd(-9, -4, -2, -1))\n", "\n", "# Exercise 4: one CiC counterfactual\n", "c0 = np.array([5, 8, 12, 15.]); c1 = np.array([6, 10, 15, 20.])\n", "q = (c0 <= 12).mean(); print(\"rank\", q, \"-> counterfactual\", np.quantile(c1, q, method=\"inverted_cdf\"), \" DiD:\", 12 + c1.mean() - c0.mean())\n", "\n", "# Exercise 13: changes-in-changes on simulated earnings\n", "rng = np.random.default_rng(72); n = 20_000\n", "U0, U1 = rng.normal(0, 1, n), rng.normal(.5, 1, n) # controls, treated (different rank distributions)\n", "h = lambda u, t: np.exp(2.5 + .4 * u + .15 * t * (1 + .3 * u)) # nonlinear, increasing in u\n", "y00, y01 = h(U0, 0), h(rng.normal(0, 1, n), 1) # controls before / after (fresh draws, same distribution)\n", "y10 = h(U1, 0); U1b = rng.normal(.5, 1, n)\n", "y11_0 = h(U1b, 1); y11 = y11_0 * (1 + .05 + .1 * (U1b - .5)) # treated after, effect grows with u\n", "cf = np.quantile(y01, np.searchsorted(np.sort(y00), y10) / n) # CiC counterfactuals for treated pre-period units\n", "did = y11.mean() - (y10.mean() + y01.mean() - y00.mean())\n", "print(f\"DiD {did:.2f} CiC mean effect {y11.mean() - cf.mean():.2f} true mean effect {(y11 - y11_0).mean():.2f}\")\n", "for p in (.1, .5, .9):\n", " print(f\"q{int(p * 100)}: CiC {np.quantile(y11, p) - np.quantile(cf, p):.2f} truth {np.quantile(y11, p) - np.quantile(y11_0, p):.2f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- DDD subtracts the DiD of an untargeted subgroup in the same states; it needs only that their difference would have trended in parallel.\n", "- CiC replaces additive parallel trends with stable rank distributions, giving quantile effects and scale invariance.\n", "- Neither can separate policies that coincide in every dimension used for comparison." ] } ] }