Day 71: Heterogeneity-Robust DiD (Callaway–Sant'Anna, Sun–Abraham)

Theory

Cohorts and group-time effects

Let $G_i$ be the period in which unit $i$ first receives treatment (its cohort), with $G_i = \infty$ for never-treated units. The building block is the group-time ATT:

ATT(g, t) = E[Y_t(g) - Y_t(\infty) \mid G = g],

the effect in period $t$ for units that adopted at $g$ .

Identification (Callaway and Sant'Anna)

Under parallel trends and no anticipation, with never-treated controls,

ATT(g, t) = E[Y_t - Y_{g-1} \mid G = g] - E[Y_t - Y_{g-1} \mid G = \infty].

Each $ATT(g, t)$ is a clean 2×2 DiD: cohort $g$ versus never-treated units, between the last pre-period $g - 1$ and period $t$ . With not-yet-treated controls, replace the second term by the change among units with $G > t$ (untreated through period $t$ ). Already-treated units are never used as controls.

Pre-treatment $ATT(g, t)$ with $t < g$ (computed relative to the preceding period, or to $g - 1$ ) serve as placebo tests.

Aggregation

Choose weights for the question asked:

Event study: $\theta(e) = \sum_g w_g\,ATT(g, g + e)$ , with $w_g$ the share of cohort $g$ among cohorts observed $e$ periods after adoption.
Overall: average of all post-treatment $ATT(g, t)$ weighted by cohort size (each treated unit-period equally), or average of the event-study $\theta(e)$ .
By cohort or calendar time as needed.

Weights are non-negative and transparent, unlike TWFE's.

Sun and Abraham's interaction-weighted estimator

A TWFE event-study regression with relative-time dummies pools cohorts; Sun and Abraham (2021) show each coefficient can be contaminated by effects at other relative times. Their fix: interact relative-time dummies with cohort dummies (using never- or last-treated units as controls), estimate cohort-specific effects $CATT(g, e)$ , then average them with cohort shares. This equals the Callaway–Sant'Anna event study with never-treated controls in the basic case.

Other robust estimators

The imputation estimator (Borusyak, Jaravel and Spiess) fits unit and time effects on untreated observations only, imputes $Y(0)$ for treated observations, and averages the differences. Covariates enter through conditional parallel trends, with outcome-regression, weighting or doubly robust versions.

Practical choices

Never-treated versus not-yet-treated controls (more data versus fewer assumptions about anticipation); balanced event-time samples (so the event study does not change composition); clustered inference across units.

Part 1 · Warm-up

Exercises 1–5 use these mean outcomes (equal cohort sizes):

Cohort	t = 1	t = 2	t = 3	t = 4
$g = 2$	10	13	15	17
$g = 3$	12	13	16	18
never	8	9	10	11

Exercise 1 · Group-time ATTs

Compute $ATT(2, 2), ATT(2, 3), ATT(2, 4), ATT(3, 3), ATT(3, 4)$ with never-treated controls.

Show solution

Step 1. Cohort 2, base period 1: $ATT(2, 2) = (13 - 10) - (9 - 8) = 2$ ; $ATT(2, 3) = (15 - 10) - (10 - 8) = 3$ ; $ATT(2, 4) = (17 - 10) - (11 - 8) = 4$ .

Step 2. Cohort 3, base period 2: $ATT(3, 3) = (16 - 13) - (10 - 9) = 2$ ; $ATT(3, 4) = (18 - 13) - (11 - 9) = 3$ .

Answer: effects of 2, 3, 4 at event times 0, 1, 2 in both cohorts.

Exercise 2 · Event-study aggregation

Compute $\theta(0)$ , $\theta(1)$ , $\theta(2)$ .

Show solution

Step 1. $\theta(0) = (ATT(2, 2) + ATT(3, 3))/2 = 2$ .

Step 2. $\theta(1) = (ATT(2, 3) + ATT(3, 4))/2 = 3$ .

Step 3. $\theta(2) = ATT(2, 4) = 4$ (only cohort 2 is observed two periods after adoption).

Exercise 3 · An overall ATT

Average all five post-treatment group-time effects with equal weight per treated cohort-period.

Show solution

Step 1. $(2 + 3 + 4 + 2 + 3)/5 = 14/5 = 2.8$ .

Answer: 2.8. (TWFE on these data gives 2.3, the code shows.)

Exercise 4 · A placebo

Compute the pre-treatment placebo for cohort 3 between periods 1 and 2.

Show solution

Step 1. $(13 - 12) - (9 - 8) = 0$ .

Answer: zero: cohort 3 and the never-treated moved in parallel before adoption.

Exercise 5 · Not-yet-treated controls

Recompute $ATT(2, 2)$ using as controls the never-treated and cohort 3 (not yet treated at $t = 2$ ), with equal weight.

Show solution

Step 1. Control change 1 → 2: never $+1$ , cohort 3 $+1$ ; average $+1$ .

Step 2. $ATT(2, 2) = 3 - 1 = 2$ .

Answer: the same here; not-yet-treated controls add data when there are few never-treated units.

Part 2 · Proofs

Exercise 6 · Identifying ATT(g, t)

Assume (i) no anticipation: $Y_s = Y_s(\infty)$ for cohort $g$ when $s < g$ ; (ii) parallel trends: $E[Y_t(\infty) - Y_{g-1}(\infty) \mid G = g] = E[Y_t(\infty) - Y_{g-1}(\infty) \mid G = \infty]$ . Prove the identification formula.

Show solution

Step 1. For cohort $g$ at $t \ge g$ : $Y_t = Y_t(g)$ ; at $g - 1$ : $Y_{g-1} = Y_{g-1}(\infty)$ by (i).

Step 2. $E[Y_t - Y_{g-1} \mid G = g] = ATT(g, t) + E[Y_t(\infty) - Y_{g-1}(\infty) \mid G = g]$ .

Step 3. For never-treated units $Y = Y(\infty)$ , so by (ii) the second term equals $E[Y_t - Y_{g-1} \mid G = \infty]$ .

Step 4. Subtract. $\blacksquare$

Exercise 7 · Not-yet-treated controls

Show the formula remains valid with controls $\{G > t\}$ if parallel trends holds for them and none of them anticipates treatment before adopting.

Show solution

Step 1. Units with $G > t$ are untreated in periods $g - 1$ and $t$ ; with no anticipation, their observed outcomes are $Y(\infty)$ in both periods.

Step 2. Parallel trends with them gives the same argument as Exercise 6, Step 3. $\blacksquare$

Exercise 8 · Anticipation contaminates not-yet-treated controls

Suppose cohort 3 changes behaviour in period 2 (one period early), raising its outcome by $a$ . Show that $ATT(2, 2)$ estimated with never-treated and cohort-3 controls (equal weights) is biased by $-a/2$ .

Show solution

Step 1. Cohort 3's observed change from 1 to 2 is its untreated trend plus $a$ .

Step 2. The average control change is the untreated trend plus $a/2$ .

Step 3. The estimate is $ATT(2, 2) - a/2$ . $\blacksquare$ With never-treated controls only, there is no bias.

Exercise 9 · Aggregations are convex combinations

Show that $\theta(e) = \sum_g w_g ATT(g, g + e)$ with $w_g = P(G = g \mid G + e \le T, G \le T)$ has non-negative weights summing to 1.

Show solution

Step 1. Conditional probabilities over the cohorts observed at event time $e$ are non-negative.

Step 2. They sum to 1 over those cohorts. $\blacksquare$ Hence $\theta(e)$ lies between the smallest and largest cohort effects at event time $e$ , unlike TWFE (Day 70, Exercise 9).

Exercise 10 · Why TWFE event studies mislead

In a staggered design, the TWFE event-study coefficient for relative time $e$ uses already-treated cohorts as part of the comparison for other cohorts. Using Day 70's Exercise 7, explain why a pre-period coefficient can be non-zero even when parallel trends holds.

Show solution

Step 1. Relative-time dummies are estimated jointly with unit and time effects; the time effects are partly identified from already-treated cohorts.

Step 2. If those cohorts' effects evolve, the time effects absorb part of their dynamics.

Step 3. Every relative-time coefficient (including leads) is then measured against these contaminated time effects, so treatment dynamics leak into the leads.

Answer: spurious pre-trends (or spurious flat pre-trends) can arise from treatment effect heterogeneity; cohort-specific estimates avoid this. $\blacksquare$

Part 3 · Applied

Exercise 11 · Staggered state paid sick-leave mandates

The table above records the share of workers with any sick leave used (%) in states adopting a mandate in period 2, in period 3, and never. Summarise the effects for a policy brief, and say which control group you would prefer.

Show solution

Step 1. Group-time effects: 2, 3, 4 points at event times 0, 1, 2 (Exercise 1).

Step 2. Event study: 2 on impact, rising to 3 after a year and 4 after two (cohort 2 only).

Step 3. Control choice: never-treated states, unless they are few or differ systematically; not-yet-treated states add precision if mandates were not anticipated.

Answer: mandates raise sick-leave use by about 2 points immediately and 4 points after two years; overall ATT 2.8.

Exercise 12 · TWFE versus group-time ATTs

On the same data, TWFE gives 2.3 while the cell-weighted ATT is 2.8. Explain the gap.

Show solution

Step 1. Effects grow with time since adoption.

Step 2. TWFE includes a comparison of cohort 3 with the already-treated cohort 2, whose effect rises from 2 to 3 to 4 during that window; the rise is subtracted.

Answer: TWFE understates the effect by about 0.5 points; the robust estimator uses only never-treated or not-yet-treated controls.

Exercise 13 · Medicaid expansions in 2014, 2015, 2016 and never

Outline the analysis of the effect of Medicaid expansion on uninsured rates with Callaway–Sant'Anna.

Show solution

Step 1. Cohorts: 2014, 2015, 2016 expanders; never-expanders as controls (or not-yet-expanded states for early years).

Step 2. Estimate $ATT(g, t)$ for every cohort and year, with pre-period placebos.

Step 3. Aggregate into an event study (balanced over, say, event times −3 to +3) and an overall ATT.

Step 4. Add baseline covariates (pre-2014 uninsured rate, income) through conditional parallel trends with a doubly robust estimator; cluster by state.

Exercise 14 · Composition changes in the event study

In Exercise 2, $\theta(2)$ uses only cohort 2. Why can this mislead, and how can it be fixed?

Show solution

Step 1. If cohorts differ in their effects, $\theta(e)$ can change across $e$ because different cohorts enter, not because effects evolve.

Step 2. Fix: restrict to a balanced event window where all cohorts used are observed at every $e$ (here $e = 0, 1$ ), or show cohort-specific profiles.

Exercise 15 · Interaction-weighted event study

Explain to a colleague how to run Sun and Abraham's estimator with standard regression software.

Show solution

Step 1. Keep never-treated (or last-treated) units as the control cohort.

Step 2. Regress the outcome on unit and time fixed effects plus interactions of each cohort with each relative-time dummy (excluding $e = -1$ ).

Step 3. Average the cohort-specific coefficients at each $e$ using each cohort's share among units observed at $e$ , and compute standard errors for these averages (delta method or bootstrap, clustered by unit).

Verify in Python

import numpy as np

Y = {2: np.array([10, 13, 15, 17.]), 3: np.array([12, 13, 16, 18.]), np.inf: np.array([8, 9, 10, 11.])}
T = 4
att = {}
for g in (2, 3):
    for t in range(g, T + 1):
        att[g, t] = (Y[g][t - 1] - Y[g][g - 2]) - (Y[np.inf][t - 1] - Y[np.inf][g - 2])
print("ATT(g,t):", {k: float(v) for k, v in att.items()})
theta = {e: np.mean([att[g, g + e] for g in (2, 3) if g + e <= T]) for e in range(0, 3)}
print("event study:", {e: float(v) for e, v in theta.items()}, " overall:", float(np.mean(list(att.values()))))
print("placebo cohort 3 (1->2):", (Y[3][1] - Y[3][0]) - (Y[np.inf][1] - Y[np.inf][0]))

# TWFE on the same cohort-level panel
M = np.array([Y[2], Y[3], Y[np.inf]]); D = np.array([[0, 1, 1, 1], [0, 0, 1, 1], [0, 0, 0, 0.]])
dm = lambda A: A - A.mean(1, keepdims=True) - A.mean(0, keepdims=True) + A.mean()
print("TWFE:", round((dm(D) * M).sum() / (dm(D) ** 2).sum(), 3))

# Unit-level simulation: many units per cohort, noise, the same dynamic effects
rng = np.random.default_rng(71); n = 300
cohorts = np.repeat([2, 3, 99], n); t = np.arange(1, T + 1)
eff = lambda e: np.where(e >= 0, 2 + e, 0)
Yu = rng.normal(0, 2, 3 * n)[:, None] + t[None, :] + eff(t[None, :] - cohorts[:, None]) + rng.normal(0, 1, (3 * n, T))
def att_gt(g, tt):
    tr, nv = cohorts == g, cohorts == 99
    return (Yu[tr, tt - 1] - Yu[tr, g - 2]).mean() - (Yu[nv, tt - 1] - Yu[nv, g - 2]).mean()
print("simulated ATT(g,t):", {(g, tt): round(float(att_gt(g, tt)), 2) for g in (2, 3) for tt in range(g, T + 1)})

Key takeaways

$ATT(g, t)$ compares each adoption cohort with never-treated or not-yet-treated units relative to its own pre-period: no forbidden comparisons.
Aggregate with explicit, non-negative weights into event-study, cohort or overall effects; keep event windows balanced.
Sun–Abraham's interaction-weighted event study and imputation estimators solve the same contamination problem in regression form.

g \ t	2	3	4	5	6	7	8	9
3	ref	2.0	2.5	3.0	3.5	4.0	4.5	5.0
5	0.0	0.0	ref	2.0	2.5	3.0	3.5	4.0
7	0.0	0.0	0.0	0.0	ref	2.0	2.5	3.0

Day 71. Heterogeneity-Robust DiD (Callaway–Sant'Anna, Sun–Abraham)

Theory

Cohorts and group-time effects

Identification (Callaway and Sant'Anna)

Aggregation

Sun and Abraham's interaction-weighted estimator

Other robust estimators

Practical choices

Part 1 · Warm-up

Exercise 1 · Group-time ATTs

Exercise 2 · Event-study aggregation

Exercise 3 · An overall ATT

Exercise 4 · A placebo

Exercise 5 · Not-yet-treated controls

Part 2 · Proofs

Exercise 6 · Identifying ATT(g, t)

Exercise 7 · Not-yet-treated controls

Exercise 8 · Anticipation contaminates not-yet-treated controls

Exercise 9 · Aggregations are convex combinations

Exercise 10 · Why TWFE event studies mislead

Part 3 · Applied

Exercise 11 · Staggered state paid sick-leave mandates

Exercise 12 · TWFE versus group-time ATTs

Exercise 13 · Medicaid expansions in 2014, 2015, 2016 and never

Exercise 14 · Composition changes in the event study

Exercise 15 · Interaction-weighted event study

Verify in Python

Key takeaways

Simulation Lab: Group-Time ATTs

g \ t	2	3	4	5	6	7	8	9
3	ref	2.0	2.5	3.0	3.5	4.0	4.5	5.0
5	0.0	0.0	ref	2.0	2.5	3.0	3.5	4.0
7	0.0	0.0	0.0	0.0	ref	2.0	2.5	3.0

g \ t	2	3	4	5	6	7	8	9
3	ref	2.0	2.5	3.0	3.5	4.0	4.5	5.0
5	0.0	0.0	ref	2.0	2.5	3.0	3.5	4.0
7	0.0	0.0	0.0	0.0	ref	2.0	2.5	3.0

g \ t	2	3	4	5	6	7	8	9
3	ref	2.0	2.5	3.0	3.5	4.0	4.5	5.0
5	0.0	0.0	ref	2.0	2.5	3.0	3.5	4.0
7	0.0	0.0	0.0	0.0	ref	2.0	2.5	3.0