{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 32: SUTVA & Interference\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "When one unit's treatment affects another's outcome, or the \"treatment\" comes in several versions. How to spot it, define new estimands, and design around it.\n", "\n", "**Learning outcomes**\n", "\n", "- State the two parts of SUTVA and recognise violations in economic and health settings.\n", "- Define direct, spillover and total effects using an exposure mapping.\n", "- Show how spillovers onto controls bias a simple difference in means.\n", "- Explain why saturation (two-stage) designs identify spillovers." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### The stable unit treatment value assumption\n", "\n", "Writing $Y_i(1)$ and $Y_i(0)$ quietly assumed two things, together called **SUTVA** (Rubin 1980):\n", "\n", "1. **No interference:** unit $i$'s outcome depends only on its own treatment, not on who else is treated.\n", "2. **No hidden versions (consistency):** there is only one version of each treatment level, so $Y_i = Y_i(T_i)$.\n", "\n", "Without (1), unit $i$ has a potential outcome for every assignment vector $\\mathbf t = (t_1, \\dots, t_n)$: $2^n$ of them. Nothing can be learned without some structure.\n", "\n", "### Where interference comes from\n", "\n", "| Mechanism | Example |\n", "|---|---|\n", "| Contagion | vaccination protects the unvaccinated (herd immunity); deworming cuts reinfection |\n", "| Information | farmers copy treated neighbours' techniques |\n", "| Competition / displacement | job-search assistance helps treated jobseekers take jobs controls would have got |\n", "| Prices (general equilibrium) | cash transfers raise local prices, affecting non-recipients |\n", "\n", "### Partial interference and exposure mappings\n", "\n", "A common, testable structure: units interact **within** groups (villages, schools, markets) but not across them, and only through the **share** of their group that is treated. Then\n", "\n", "$$\n", "Y_{ig} = Y_{ig}(t_{ig},\\, s_g), \\qquad s_g = \\text{saturation (share treated in group } g).\n", "$$\n", "\n", "Effects at saturation $s$ (averages over units):\n", "\n", "- **Direct effect:** $\\text{DE}(s) = E[Y(1, s) - Y(0, s)]$ (own treatment, holding others' fixed).\n", "- **Spillover (indirect) effect:** $\\text{SE}(s) = E[Y(0, s) - Y(0, 0)]$ (on the untreated, of others being treated).\n", "- **Total effect:** $E[Y(1, s) - Y(0, 0)] = \\text{DE}(s) + \\text{SE}(s)$.\n", "\n", "### Designs\n", "\n", "- **Cluster randomisation** (Day 40) puts the whole group in one arm, so spillovers stay inside: estimates the total policy effect.\n", "- **Saturation (two-stage) designs** first randomise groups to saturations (0%, 50%, 100%…), then individuals within groups (Hudgens & Halloran 2008; Baird et al. 2018). Comparing untreated units in treated groups with pure controls identifies spillovers.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · How many potential outcomes?\n", "\n", "A village has 3 farmers. Without SUTVA, how many potential outcomes does each farmer have? With SUTVA?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Without no-interference, each farmer's yield can depend on all three treatments: $2^3 = 8$ assignment vectors.\n", "\n", "**Step 2.** With SUTVA, only own treatment matters: 2.\n", "\n", "**Answer:** 8 versus 2. With 1,000 farmers it would be $2^{1000}$ versus 2, which is why structure is essential.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Spot the violation\n", "\n", "For each case, say whether SUTVA fails and which part: (a) a measles vaccine; (b) a training programme run by two NGOs of different quality; (c) fertiliser applied to a farmer's own field.\n", "\n", "
Show solution\n", "\n", "**Step 1.** (a) Vaccinating others lowers my infection risk: **interference**.\n", "\n", "**Step 2.** (b) \"Training\" is really two treatments: **hidden versions**.\n", "\n", "**Step 3.** (c) Fertiliser on my plot affects my yield only (unless it runs off into neighbours' fields): SUTVA is plausible.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · An exposure mapping\n", "\n", "In a village of 10 households, 4 are treated, including household $i$. What are $i$'s own treatment and the share of *other* households treated?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Own treatment $t_i = 1$.\n", "\n", "**Step 2.** Other treated households: $4 - 1 = 3$ of the other 9: share $3/9 = 1/3$.\n", "\n", "**Answer:** exposure $(1, 1/3)$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · Direct effect\n", "\n", "At 50% saturation, treated households average 120 (consumption, \\$/month) and untreated households in the same villages average 100. What is the direct effect at $s = 0.5$?\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\text{DE}(0.5) = E[Y(1, 0.5)] - E[Y(0, 0.5)]$, both measured in the same villages.\n", "\n", "**Step 2.** $120 - 100 = 20$.\n", "\n", "**Answer:** \\$20/month (assuming individuals were randomised within villages).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Spillover effect\n", "\n", "Untreated households in pure-control villages average 92. With Exercise 4, compute the spillover and total effects.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Spillover $= E[Y(0, 0.5)] - E[Y(0, 0)] = 100 - 92 = 8$.\n", "\n", "**Step 2.** Total $= E[Y(1, 0.5)] - E[Y(0, 0)] = 120 - 92 = 28 = 20 + 8$.\n", "\n", "**Answer:** spillover \\$8, total \\$28.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · SUTVA collapses the outcomes\n", "\n", "Show that under no interference $Y_i(\\mathbf t) = Y_i(t_i)$, and that consistency then gives $Y_i = T_iY_i(1) + (1 - T_i)Y_i(0)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** No interference says $Y_i(\\mathbf t) = Y_i(\\mathbf t')$ whenever $t_i = t'_i$, so $Y_i(\\mathbf t)$ depends on $\\mathbf t$ only through $t_i$: write it $Y_i(t_i)$.\n", "\n", "**Step 2.** Consistency says the observed outcome equals the potential outcome of the treatment received: $Y_i = Y_i(T_i)$.\n", "\n", "**Step 3.** If $T_i = 1$, $Y_i = Y_i(1)$; if $T_i = 0$, $Y_i = Y_i(0)$. Both cases are $T_iY_i(1) + (1 - T_i)Y_i(0)$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · Within-group randomisation identifies the direct effect\n", "\n", "Assume partial interference with exposure $(t, s)$. In groups assigned saturation $s$, individuals are treated completely at random. Show $E[Y \\mid T = 1, s] - E[Y \\mid T = 0, s] = \\text{DE}(s)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** In a saturation-$s$ group, every unit has exposure $(T_i, s)$, so $Y_i = Y_i(T_i, s)$.\n", "\n", "**Step 2.** Within those groups, $T_i$ is randomised, so it is independent of $(Y_i(1, s), Y_i(0, s))$.\n", "\n", "**Step 3.** Hence $E[Y \\mid T = 1, s] = E[Y(1, s)]$ and $E[Y \\mid T = 0, s] = E[Y(0, s)]$ (Day 6, Exercise 14).\n", "\n", "**Step 4.** The difference is $\\text{DE}(s)$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Total = direct + spillover\n", "\n", "Prove $E[Y(1, s) - Y(0, 0)] = \\text{DE}(s) + \\text{SE}(s)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Add and subtract $Y(0, s)$: $Y(1, s) - Y(0, 0) = [Y(1, s) - Y(0, s)] + [Y(0, s) - Y(0, 0)]$.\n", "\n", "**Step 2.** Take expectations (linearity): $\\text{DE}(s) + \\text{SE}(s)$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Contaminated controls\n", "\n", "Individuals are randomised within every village at saturation $s$, and the analyst treats the within-village difference in means as \"the effect\". Show it equals the total effect minus the spillover on controls.\n", "\n", "
Show solution\n", "\n", "**Step 1.** By Exercise 7 the comparison estimates $E[Y(1, s)] - E[Y(0, s)]$.\n", "\n", "**Step 2.** Rewrite using Exercise 8: $E[Y(1, s)] - E[Y(0, s)] = \\underbrace{E[Y(1, s) - Y(0, 0)]}_{\\text{total}} - \\underbrace{E[Y(0, s) - Y(0, 0)]}_{\\text{spillover on controls}}$.\n", "\n", "**Answer:** positive spillovers (herd immunity) make the within-village comparison *understate* the policy's total benefit; negative spillovers (displacement) make it *overstate* it.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Hidden versions average out\n", "\n", "A \"treatment\" has two versions, $a$ and $b$, delivered with probabilities $\\pi$ and $1 - \\pi$ independently of potential outcomes. Show the difference in means estimates $\\pi\\,\\text{ATE}_a + (1 - \\pi)\\,\\text{ATE}_b$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** A treated unit's outcome is $Y_i(a)$ with probability $\\pi$ and $Y_i(b)$ otherwise, so $E[Y \\mid T = 1] = \\pi E[Y(a)] + (1 - \\pi)E[Y(b)]$.\n", "\n", "**Step 2.** $E[Y \\mid T = 0] = E[Y(0)]$ by randomisation.\n", "\n", "**Step 3.** Difference: $\\pi(E[Y(a)] - E[Y(0)]) + (1 - \\pi)(E[Y(b)] - E[Y(0)])$. $\\blacksquare$\n", "\n", "The estimate is a mixture: it describes *this* blend of versions, not either version alone.\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Deworming and reinfection\n", "\n", "In a school deworming trial, infection rates are: treated pupils in treatment schools 0.20; untreated pupils in treatment schools 0.35; pupils in control schools 0.50. Compute the direct, spillover and total effects, and what a within-school comparison would report.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Direct effect (within treatment schools): $0.20 - 0.35 = -0.15$.\n", "\n", "**Step 2.** Spillover on untreated pupils: $0.35 - 0.50 = -0.15$.\n", "\n", "**Step 3.** Total effect for treated pupils: $0.20 - 0.50 = -0.30$.\n", "\n", "**Step 4.** A within-school comparison reports the direct effect, −0.15.\n", "\n", "**Answer:** the within-school comparison captures only half the benefit; deworming also protects untreated classmates. This was a central finding of Miguel & Kremer (2004).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Job-search assistance and displacement\n", "\n", "Job-finding rates within 6 months: assisted jobseekers 0.40; unassisted jobseekers in labour markets where half were assisted 0.28; jobseekers in untreated markets 0.32. Compute the direct and spillover effects, and the net effect on the average jobseeker in a 50%-assisted market.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Direct effect: $0.40 - 0.28 = 0.12$.\n", "\n", "**Step 2.** Spillover on the unassisted: $0.28 - 0.32 = -0.04$ (they lose jobs to assisted competitors).\n", "\n", "**Step 3.** Average job-finding in a 50%-assisted market: $0.5(0.40) + 0.5(0.28) = 0.34$.\n", "\n", "**Step 4.** Net effect versus untreated markets: $0.34 - 0.32 = 0.02$.\n", "\n", "**Answer:** the direct effect (12 points) vastly overstates the 2-point net gain: most of the \"effect\" is reshuffling who gets the jobs (as in Crépon et al., 2013).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Herd immunity\n", "\n", "When 80% of a town is vaccinated, the infection risk of *unvaccinated* residents falls from 0.10 to 0.04. The vaccine reduces a vaccinated person's risk to 0.01 at that coverage. Compute the indirect and direct effects at 80% coverage.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Indirect (spillover) effect on the unvaccinated: $0.04 - 0.10 = -0.06$.\n", "\n", "**Step 2.** Direct effect at 80% coverage: $0.01 - 0.04 = -0.03$.\n", "\n", "**Step 3.** Total effect for a vaccinated person versus no programme: $0.01 - 0.10 = -0.09 = -0.03 - 0.06$.\n", "\n", "**Answer:** at high coverage, two-thirds of a vaccinated person's protection comes from others being vaccinated. A trial randomising individuals at low coverage would miss this.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Cash transfers and local prices\n", "\n", "A large cash transfer to half the households in each village raises local food prices by 5%. Which SUTVA component fails, who is affected, and how should the experiment be designed?\n", "\n", "
Show solution\n", "\n", "**Step 1.** No interference fails: non-recipients face higher prices, a (negative) spillover through markets.\n", "\n", "**Step 2.** Recipients' measured gains are also net of the price rise.\n", "\n", "**Step 3.** Design: randomise at the market (village or cluster-of-villages) level, ideally with varying saturations, and measure prices directly.\n", "\n", "**Answer:** a within-village comparison mis-states welfare effects; market-level randomisation captures the general-equilibrium effect (as in Egger et al., 2022).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Two training providers\n", "\n", "A national training programme is delivered by provider A (effect +\\$3,000) to 40% of trainees and provider B (effect +\\$1,000) to 60%, assigned independently of workers' potential earnings. What does the evaluation estimate, and what would scaling up through provider B alone deliver?\n", "\n", "
Show solution\n", "\n", "**Step 1.** By Exercise 10: $0.4(3{,}000) + 0.6(1{,}000) = 1{,}200 + 600 = 1{,}800$.\n", "\n", "**Step 2.** A scale-up through B only would deliver \\$1,000 per trainee.\n", "\n", "**Answer:** the evaluation reports \\$1,800, which overstates the scale-up effect by 80%. Record which version each unit received; \"the treatment\" must match the policy being considered.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "rng = np.random.default_rng(32)\n", "G, m = 400, 20 # villages, households per village\n", "sat = rng.choice([0.0, 0.5], G) # half the villages untreated, half 50% treated\n", "DE, SPILL = 20.0, 8.0 # direct effect and spillover at 50% saturation\n", "rows = []\n", "for g in range(G):\n", " T = np.zeros(m, bool)\n", " T[rng.choice(m, int(sat[g] * m), replace=False)] = True\n", " y = 92 + DE * T + SPILL * (sat[g] > 0) + rng.normal(0, 15, m) # everyone in a treated village gets the spillover\n", " rows += [(sat[g], t, v) for t, v in zip(T, y)]\n", "s, t, y = map(np.array, zip(*rows))\n", "direct = y[(s == 0.5) & t].mean() - y[(s == 0.5) & ~t].mean()\n", "spill = y[(s == 0.5) & ~t].mean() - y[s == 0].mean()\n", "total = y[(s == 0.5) & t].mean() - y[s == 0].mean()\n", "print(f\"direct {direct:.1f} (truth 20), spillover {spill:.1f} (truth 8), total {total:.1f} (truth 28)\")\n", "\n", "# Job-search displacement (Exercise 12)\n", "print(\"net effect in a 50% market:\", round(0.5 * 0.40 + 0.5 * 0.28 - 0.32, 3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- SUTVA = no interference + no hidden versions; without it, \"the\" effect is not even defined.\n", "- With spillovers, distinguish direct, spillover and total effects; within-group comparisons capture only the direct effect.\n", "- Saturation designs and cluster randomisation turn spillovers from a nuisance into an estimand." ] } ] }