{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 68: Parallel Trends & Event Studies\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "With several periods, an event-study regression traces effects period by period and puts the parallel-trends assumption on display through pre-treatment coefficients. Why pre-trend tests have low power, how anticipation shifts the reference period, and how to report results that are robust to bounded violations.\n", "\n", "**Learning outcomes**\n", "\n", "- Specify and interpret an event-study regression with unit and time fixed effects and a reference period.\n", "- Compute event-study coefficients from group-by-period means.\n", "- Calculate the power of a pre-trend test and explain why passing it is weak evidence.\n", "- Construct sensitivity intervals that allow post-treatment violations bounded by pre-treatment ones." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### The event-study regression\n", "\n", "With one treated group adopting at date $E$ and many periods, define event time $k = t - E$. The event-study specification is\n", "\n", "$$\n", "Y_{it} = \\alpha_i + \\lambda_t + \\sum_{k \\ne -1}\\beta_k\\,\\mathbf 1[t - E = k]\\,G_i + \\varepsilon_{it},\n", "$$\n", "\n", "with unit fixed effects $\\alpha_i$, period fixed effects $\\lambda_t$, and the period just before treatment ($k = -1$) omitted as the **reference**. Each $\\beta_k$ is a DiD comparing period $k$ with period $-1$:\n", "\n", "$$\n", "\\beta_k = (\\bar Y_{G=1, k} - \\bar Y_{G=1, -1}) - (\\bar Y_{G=0, k} - \\bar Y_{G=0, -1}).\n", "$$\n", "\n", "- $k \\ge 0$: dynamic treatment effects (ATT $k$ periods after adoption), under parallel trends.\n", "- $k < 0$: **placebo** or **pre-trend** coefficients, zero if trends were parallel before treatment.\n", "\n", "### Reading pre-trends\n", "\n", "Flat pre-period coefficients are reassuring, but:\n", "\n", "1. **Low power.** Pre-trend coefficients are noisy; a violation large enough to matter can easily go undetected (Exercise 12).\n", "2. **Pre-testing distortion.** Reporting only studies that pass a pre-test selects samples where noise happened to hide the trend, which can increase bias (Roth, 2022).\n", "3. **Parallel pre-trends do not guarantee parallel post-trends.** The assumption is about the counterfactual after treatment.\n", "\n", "### Remedies and sensitivity\n", "\n", "- **Group-specific linear trends:** fit a line through the pre-period differences and extrapolate. Unbiased if the violation is exactly linear; can absorb part of a dynamic effect if not.\n", "- **Bounded violations (Rambachan and Roth):** assume the post-treatment deviation from parallel trends is at most $M$ times the largest pre-treatment deviation, and report the resulting interval for each $M$, plus the **breakdown** value of $M$ at which the effect is no longer significant.\n", "- **Conditional parallel trends:** assume parallel trends only among units with the same pre-treatment covariates; estimate with regression, weighting or doubly robust DiD.\n", "\n", "### Anticipation\n", "\n", "If behaviour changes one period before adoption, the coefficient at $k = -1$ is already affected. Use $k = -2$ (or earlier) as the reference.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Reading coefficients\n", "\n", "An event study reports $\\hat\\beta_{-3} = 0.1$, $\\hat\\beta_{-2} = -0.05$, $\\hat\\beta_0 = 1.2$, $\\hat\\beta_1 = 1.8$, $\\hat\\beta_2 = 2.1$ (SEs around 0.2). Describe the results.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Pre-period coefficients are small and insignificant: no visible pre-trend.\n", "\n", "**Step 2.** The effect appears on impact (1.2) and grows over two years (2.1).\n", "\n", "**Answer:** a dynamic effect that builds over time, with parallel pre-trends as supporting (not conclusive) evidence.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · The reference period\n", "\n", "Why must one event-time indicator be omitted, and what does choosing $k = -1$ mean?\n", "\n", "
Show solution\n", "\n", "**Step 1.** With unit and time fixed effects, the full set of event-time dummies for the treated group is collinear with the unit fixed effect.\n", "\n", "**Step 2.** Omitting $k = -1$ measures every $\\beta_k$ relative to the last pre-treatment period.\n", "\n", "**Answer:** all coefficients are DiDs relative to period −1, which is zero by construction.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · Coefficients from means\n", "\n", "Treated means in periods −3, −2, −1, 0, 1: 10, 11, 12, 16, 17. Comparison: 8, 9, 10, 11, 12. Compute $\\beta_{-3}, \\beta_{-2}, \\beta_0, \\beta_1$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\beta_{-3} = (10 - 12) - (8 - 10) = 0$; $\\beta_{-2} = (11 - 12) - (9 - 10) = 0$.\n", "\n", "**Step 2.** $\\beta_0 = (16 - 12) - (11 - 10) = 3$.\n", "\n", "**Step 3.** $\\beta_1 = (17 - 12) - (12 - 10) = 3$.\n", "\n", "**Answer:** no pre-trend, a constant effect of 3.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · Linear extrapolation\n", "\n", "Pre-period coefficients are $\\beta_{-3} = -0.6$, $\\beta_{-2} = -0.3$ (and $\\beta_{-1} = 0$): a linear trend of +0.3 per period. If $\\hat\\beta_2 = 1.5$, what is the trend-adjusted effect at $k = 2$?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Extrapolated violation at $k = 2$: $0.3 \\times 2 = 0.6$.\n", "\n", "**Step 2.** Adjusted: $1.5 - 0.6 = 0.9$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Joint versus individual tests\n", "\n", "Why test pre-period coefficients jointly rather than one at a time?\n", "\n", "
Show solution\n", "\n", "**Step 1.** With several pre-periods, one coefficient may be significant by chance (multiple testing), while several small same-sign coefficients may each be insignificant yet jointly informative.\n", "\n", "**Answer:** a joint F (Wald) test uses all pre-period information at once with correct size.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · Event-study coefficients identify dynamic ATTs\n", "\n", "With one treated and one comparison group, balanced data and the saturated event-study regression, prove $\\hat\\beta_k$ equals the DiD of means between periods $k$ and $-1$, and that under parallel trends and no anticipation it is unbiased for $ATT_k$ ($k \\ge 0$) and for 0 ($k < 0$).\n", "\n", "
Show solution\n", "\n", "**Step 1.** The regression has one parameter per group-period cell (unit effects absorb group levels, period effects the comparison path, $\\beta_k$ the treated deviations); it is saturated in group × period means.\n", "\n", "**Step 2.** So fitted cell means equal sample cell means, and $\\hat\\beta_k$ is the treated-minus-comparison change from period $-1$ to $k$.\n", "\n", "**Step 3.** By Day 67, Exercise 9, its expectation is $ATT_k + \\Delta_k$, where $\\Delta_k$ is the difference in untreated trends between $-1$ and $k$; under parallel trends $\\Delta_k = 0$.\n", "\n", "**Step 4.** For $k < 0$ the ATT is 0 (no anticipation), so $E[\\hat\\beta_k] = 0$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · Linear trend adjustment\n", "\n", "Suppose the violation is linear: $\\Delta_k = s\\,(k + 1)$ for all $k$. Show that $\\hat\\beta_k - \\hat s(k + 1)$, with $\\hat s$ fitted on pre-periods, is unbiased for $ATT_k$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Pre-period coefficients have expectation $s(k + 1)$, so a least-squares slope through them (and the origin at $k = -1$) is unbiased for $s$.\n", "\n", "**Step 2.** $E[\\hat\\beta_k] = ATT_k + s(k + 1)$ for $k \\ge 0$.\n", "\n", "**Step 3.** Subtract $E[\\hat s](k + 1) = s(k + 1)$. $\\blacksquare$ If the violation is not linear, the adjustment is biased.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Power of a pre-trend test\n", "\n", "A pre-period coefficient has expectation $\\mu$ and standard error $\\sigma$ (normal). Show the power of a two-sided 5% test is $\\Phi(-1.96 - \\mu/\\sigma) + 1 - \\Phi(1.96 - \\mu/\\sigma)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** The t-statistic is $N(\\mu/\\sigma, 1)$.\n", "\n", "**Step 2.** Rejection: $|t| > 1.96$.\n", "\n", "**Step 3.** $P(t > 1.96) = 1 - \\Phi(1.96 - \\mu/\\sigma)$ and $P(t < -1.96) = \\Phi(-1.96 - \\mu/\\sigma)$. Add. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Bounded violations\n", "\n", "Assume the post-period violation satisfies $|\\Delta_{post}| \\le M\\bar\\Delta_{pre}$, where $\\bar\\Delta_{pre}$ is the largest absolute pre-period deviation (treated as known). Show that $ATT \\in [\\hat\\beta - M\\bar\\Delta_{pre},\\; \\hat\\beta + M\\bar\\Delta_{pre}]$ in population, and give a 95% version.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\beta = ATT + \\Delta_{post}$, so $ATT = \\beta - \\Delta_{post}$.\n", "\n", "**Step 2.** With $\\Delta_{post} \\in [-M\\bar\\Delta_{pre}, M\\bar\\Delta_{pre}]$, $ATT \\in [\\beta - M\\bar\\Delta_{pre}, \\beta + M\\bar\\Delta_{pre}]$.\n", "\n", "**Step 3.** Accounting for sampling error in $\\hat\\beta$: $[\\hat\\beta - M\\bar\\Delta_{pre} - 1.96\\,\\operatorname{SE},\\; \\hat\\beta + M\\bar\\Delta_{pre} + 1.96\\,\\operatorname{SE}]$ covers the ATT with probability at least 95%. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Anticipation and the reference period\n", "\n", "Suppose units react one period early, so $E[\\hat\\beta_{-1}]$ would be $a \\ne 0$ if it were estimated. Show that normalising $\\beta_{-1} = 0$ shifts every other coefficient by $-a$, and that using $k = -2$ as the reference removes the bias.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Each $\\hat\\beta_k$ is a DiD relative to period −1: $E[\\hat\\beta_k] = (ATT_k + \\Delta_k) - (a + \\Delta_{-1})$; with parallel trends $\\Delta = 0$, it is $ATT_k - a$.\n", "\n", "**Step 2.** With $k = -2$ as reference (no anticipation there), $E[\\hat\\beta_k] = ATT_k$, and the coefficient at $k = -1$ estimates $a$. $\\blacksquare$\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · State minimum wage and teen employment\n", "\n", "Teen employment rates (%) in a state that raised its minimum wage in period 0, periods −3 to 2: 30.0, 30.5, 31.0, 30.0, 29.5, 29.3. In comparison states: 32.0, 32.4, 32.9, 33.2, 33.5, 33.7. Compute the event-study coefficients (reference −1).\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\beta_{-3} = (30.0 - 31.0) - (32.0 - 32.9) = -1.0 + 0.9 = -0.1$.\n", "\n", "**Step 2.** $\\beta_{-2} = (30.5 - 31.0) - (32.4 - 32.9) = -0.5 + 0.5 = 0$.\n", "\n", "**Step 3.** $\\beta_0 = (30.0 - 31.0) - (33.2 - 32.9) = -1.3$.\n", "\n", "**Step 4.** $\\beta_1 = (29.5 - 31.0) - (33.5 - 32.9) = -2.1$; $\\beta_2 = (29.3 - 31.0) - (33.7 - 32.9) = -2.5$.\n", "\n", "**Answer:** flat pre-trends and a growing employment decline of 1.3 to 2.5 points.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · How likely is a pre-test to catch a trend?\n", "\n", "In a study of hospital mergers on prices, a linear violation would make $E[\\hat\\beta_{-3}] = -0.9$ (percent), with SE 0.5. What is the probability the pre-trend test at $k = -3$ rejects?\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\mu/\\sigma = -1.8$.\n", "\n", "**Step 2.** Power $= \\Phi(-1.96 + 1.8) + 1 - \\Phi(1.96 + 1.8) = \\Phi(-0.16) + 1 - \\Phi(3.76) \\approx 0.436 + 0.000 = 0.44$.\n", "\n", "**Answer:** about 44%: more often than not, this sizeable trend goes undetected.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Sensitivity for a paid-leave mandate\n", "\n", "A paid-family-leave mandate raises mothers' employment by $\\hat\\beta = 2.0$ points (SE 0.5) one year after; the largest pre-period deviation is 0.4. Compute the 95% sensitivity intervals for $M = 1$ and $M = 2$, and the breakdown value of $M$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $M = 1$: $[2.0 - 0.4 - 0.98,\\; 2.0 + 0.4 + 0.98] = [0.62, 3.38]$.\n", "\n", "**Step 2.** $M = 2$: $[2.0 - 0.8 - 0.98,\\; 2.0 + 0.8 + 0.98] = [0.22, 3.78]$.\n", "\n", "**Step 3.** Breakdown: lower bound $2.0 - 0.4M - 0.98 = 0 \\Rightarrow M = 1.02/0.4 = 2.55$.\n", "\n", "**Answer:** the effect stays positive unless post-period violations are more than 2.5 times the largest pre-period one.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Anticipation in hospital mergers\n", "\n", "Hospitals announce a merger a year before it closes and start renegotiating contracts. An event study normalised at $k = -1$ shows $\\hat\\beta_0 = 3\\%$ and $\\hat\\beta_{-2} = -2\\%$. Reinterpret with $k = -2$ as the reference.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Re-normalising to $k = -2$ adds 2 points to every coefficient: $\\beta_{-1} = 2\\%$ (anticipation effect), $\\beta_0 = 5\\%$.\n", "\n", "**Answer:** prices rose 2% already after the announcement and 5% by closing; the original normalisation understated the effect.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Conditional parallel trends\n", "\n", "States that expanded Medicaid had lower baseline uninsured rates, and uninsured rates predict mortality trends. How can covariates restore parallel trends, and what is assumed?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Assume parallel trends conditional on baseline covariates $X$ (uninsured rate, income, age structure): $E[\\Delta Y(0) \\mid G = 1, X] = E[\\Delta Y(0) \\mid G = 0, X]$.\n", "\n", "**Step 2.** Estimate by comparing changes within $X$ cells, by outcome regression of $\\Delta Y$ on $X$ in comparison states, or by doubly robust DiD (propensity weighting plus outcome regression).\n", "\n", "**Step 3.** Use only pre-treatment covariates: post-treatment ones may be affected by the expansion (Day 53).\n", "\n", "**Answer:** conditional parallel trends plus overlap in $X$.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "from scipy.stats import norm\n", "\n", "def es(T, C, ref):\n", " \"\"\"Event-study coefficients from treated and comparison means; ref = index of period -1.\"\"\"\n", " return np.round([(T[k] - T[ref]) - (C[k] - C[ref]) for k in range(len(T))], 3)\n", "print(\"Ex 3:\", es([10, 11, 12, 16, 17], [8, 9, 10, 11, 12], 2))\n", "print(\"Ex 11:\", es([30, 30.5, 31, 30, 29.5, 29.3], [32, 32.4, 32.9, 33.2, 33.5, 33.7], 2))\n", "m = -.9 / .5\n", "print(\"pre-test power:\", round(norm.cdf(-1.96 - m) + 1 - norm.cdf(1.96 - m), 3))\n", "for M in (1, 2):\n", " print(f\"M={M}: [{2 - .4 * M - .98:.2f}, {2 + .4 * M + .98:.2f}]\", end=\" \")\n", "print(\"breakdown M:\", round((2 - .98) / .4, 2))\n", "\n", "# Event-study regression with unit and time fixed effects on a simulated panel (linear pre-trend 0.3)\n", "rng = np.random.default_rng(68)\n", "N, Tn, E = 200, 8, 5\n", "G = (np.arange(N) < N // 2).astype(float)\n", "t = np.arange(Tn); k = t - E\n", "Y = (rng.normal(0, 2, N)[:, None] + 0.5 * t[None, :] + 0.3 * k[None, :] * G[:, None]\n", " + G[:, None] * np.where(k >= 0, 1 + 0.5 * k, 0)[None, :] + rng.normal(0, 1, (N, Tn)))\n", "rows = []\n", "for i in range(N):\n", " for s in range(Tn):\n", " r = [1.0] + [float(i == j) for j in range(1, N)] + [float(s == q) for q in range(1, Tn)]\n", " r += [G[i] * float(k[s] == kk) for kk in range(-E, Tn - E) if kk != -1]\n", " rows.append(r)\n", "b = np.linalg.lstsq(np.array(rows), Y.ravel(), rcond=None)[0][-(Tn - 1):]\n", "labels = [kk for kk in range(-E, Tn - E) if kk != -1]\n", "print(\"event-study betas:\", {kk: round(float(v), 2) for kk, v in zip(labels, b)})\n", "print(\"truth (effect + 0.3*(k+1) trend):\", {kk: round((1 + .5 * kk if kk >= 0 else 0) + .3 * (kk + 1), 2) for kk in labels})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- Event-study coefficients are DiDs relative to a reference period; pre-period coefficients test for diverging trends before treatment.\n", "- Pre-trend tests have limited power, and passing them does not establish parallel trends after treatment.\n", "- Report sensitivity to bounded violations (and the breakdown value), re-normalise if there is anticipation, and use conditional parallel trends when observable differences predict trends." ] } ] }