{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 41: Non-Compliance & Intention-to-Treat\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "When people do not take the treatment they are assigned. Intention-to-treat effects, compliance types, and the effect for compliers.\n", "\n", "**Learning outcomes**\n", "\n", "- Define assignment, receipt, compliance types and the intention-to-treat effects.\n", "- Prove that, under exclusion and monotonicity, the ITT on outcomes equals the complier share times the complier effect.\n", "- Show why as-treated and per-protocol comparisons are biased.\n", "- Compute and interpret ITT and complier effects in training, health and encouragement designs." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### Assignment is not receipt\n", "\n", "In many experiments the researcher randomises an **offer** $Z$ (a voucher, an invitation, an encouragement letter), but people decide whether to take up the **treatment** $D$. Each unit has potential take-up $D(1), D(0)$ and potential outcomes. Four **compliance types**:\n", "\n", "| Type | $D(1)$ | $D(0)$ |\n", "|---|---|---|\n", "| Complier | 1 | 0 |\n", "| Always-taker | 1 | 1 |\n", "| Never-taker | 0 | 0 |\n", "| Defier | 0 | 1 |\n", "\n", "Types are defined by potential take-up, so they are fixed before assignment and unaffected by it.\n", "\n", "### Intention-to-treat\n", "\n", "Random assignment of $Z$ identifies two **intention-to-treat** effects:\n", "\n", "$$\n", "\\text{ITT}_D = E[D \\mid Z = 1] - E[D \\mid Z = 0], \\qquad \\text{ITT}_Y = E[Y \\mid Z = 1] - E[Y \\mid Z = 0].\n", "$$\n", "\n", "$\\text{ITT}_Y$ is the effect of *offering* the programme: often exactly the policy-relevant quantity (governments can offer, not compel).\n", "\n", "### The effect for compliers\n", "\n", "Assume (i) $Z$ is randomised; (ii) **exclusion**: $Z$ affects $Y$ only through $D$; (iii) **monotonicity**: no defiers; (iv) $\\text{ITT}_D > 0$. Then\n", "\n", "$$\n", "\\text{CACE} = E[Y(1) - Y(0) \\mid \\text{complier}] = \\frac{\\text{ITT}_Y}{\\text{ITT}_D}.\n", "$$\n", "\n", "This is the Wald (IV) estimator (Days 12 and 19); Day 63 develops it as the LATE. With **one-sided non-compliance** (no one assigned to control can get treated), there are no always-takers, so compliers are exactly the treated and CACE = ATT (Bloom 1984).\n", "\n", "### What not to do\n", "\n", "- **As-treated:** compare those who took the treatment with those who did not, ignoring $Z$. This is an observational comparison with self-selection.\n", "- **Per-protocol:** compare those who followed their assignment in each arm. The arms then contain different types (e.g. compliers + always-takers versus compliers + never-takers), so randomisation is lost.\n", "\n", "## Part 1 · Warm-up\n", "\n", "A job-training offer is randomised. Employment is 45% among those offered and 40% among those not offered. Take-up is 60% among the offered and 10% among the non-offered (who found similar training elsewhere)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · ITT on employment\n", "\n", "Compute $\\text{ITT}_Y$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\text{ITT}_Y = 0.45 - 0.40 = 0.05$.\n", "\n", "**Answer:** offering training raises employment by 5 points.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · ITT on take-up\n", "\n", "Compute $\\text{ITT}_D$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\text{ITT}_D = 0.60 - 0.10 = 0.50$.\n", "\n", "**Answer:** the offer raises training participation by 50 points.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · The complier effect\n", "\n", "Compute the CACE.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\text{CACE} = 0.05/0.50 = 0.10$.\n", "\n", "**Answer:** training raises employment by 10 points for workers who train only when offered.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · Type shares\n", "\n", "Assuming no defiers, find the shares of always-takers, compliers and never-takers.\n", "\n", "
Show solution\n", "\n", "**Step 1.** In the control arm, those who train are always-takers: 10%.\n", "\n", "**Step 2.** In the offer arm, those who do not train are never-takers: $1 - 0.60 = 40\\%$.\n", "\n", "**Step 3.** Compliers: $1 - 0.10 - 0.40 = 50\\%$ (equal to $\\text{ITT}_D$).\n", "\n", "**Answer:** 10% always-takers, 50% compliers, 40% never-takers.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · One-sided non-compliance\n", "\n", "A vocational programme can only be accessed with the voucher; 70% of voucher winners enrol. What are $\\text{ITT}_D$ and the share of always-takers?\n", "\n", "
Show solution\n", "\n", "**Step 1.** No one without a voucher can enrol: $E[D \\mid Z = 0] = 0$, so no always-takers.\n", "\n", "**Step 2.** $\\text{ITT}_D = 0.70 - 0 = 0.70$.\n", "\n", "**Answer:** 0.70; always-takers 0%; so CACE = ATT (Exercise 7).\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · ITT = complier share × CACE\n", "\n", "Under randomisation, exclusion and no defiers, prove $\\text{ITT}_Y = \\pi_c\\,\\text{CACE}$ and $\\text{ITT}_D = \\pi_c$, where $\\pi_c$ is the complier share.\n", "\n", "
Show solution\n", "\n", "**Step 1.** By exclusion, write each unit's outcome under assignment $z$ as $Y(D(z))$. Then $Y_i(Z{=}1) - Y_i(Z{=}0) = Y_i(D_i(1)) - Y_i(D_i(0))$.\n", "\n", "**Step 2.** By type: always-takers and never-takers have $D(1) = D(0)$, so their difference is 0. Compliers have $D(1) = 1$, $D(0) = 0$, so their difference is $Y_i(1) - Y_i(0)$. There are no defiers.\n", "\n", "**Step 3.** Randomisation makes $\\text{ITT}_Y$ the average of these differences: $\\text{ITT}_Y = \\pi_c\\,E[Y(1) - Y(0) \\mid \\text{complier}] = \\pi_c\\,\\text{CACE}$.\n", "\n", "**Step 4.** Same argument for take-up: $D(1) - D(0)$ is 1 for compliers and 0 otherwise, so $\\text{ITT}_D = \\pi_c$.\n", "\n", "**Step 5.** Divide: $\\text{CACE} = \\text{ITT}_Y/\\text{ITT}_D$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · Bloom's result\n", "\n", "With one-sided non-compliance, prove CACE = ATT.\n", "\n", "
Show solution\n", "\n", "**Step 1.** No one assigned to control can be treated, so $D(0) = 0$ for all: there are no always-takers or defiers.\n", "\n", "**Step 2.** The treated ($D = 1$) are exactly those with $Z = 1$ and $D(1) = 1$: compliers assigned to treatment.\n", "\n", "**Step 3.** Randomisation makes compliers assigned to treatment representative of all compliers, so $E[\\tau \\mid D = 1] = E[\\tau \\mid \\text{complier}]$.\n", "\n", "**Answer:** ATT = CACE $= \\text{ITT}_Y/\\text{ITT}_D$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Per-protocol is biased\n", "\n", "In Exercise 4's population, show that the per-protocol comparison (offered who trained versus not-offered who did not train) compares different mixtures of types.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Offered who trained: compliers and always-takers, in proportions $0.5 : 0.1$.\n", "\n", "**Step 2.** Not offered who did not train: compliers and never-takers, $0.5 : 0.4$.\n", "\n", "**Step 3.** If always-takers are more motivated and never-takers less, the first group would have higher employment even without training.\n", "\n", "**Answer:** per-protocol mixes the training effect with differences between types; only the comparison by assignment ($Z$) is protected by randomisation.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · As-treated is an observational study\n", "\n", "Show that the as-treated difference $E[Y \\mid D = 1] - E[Y \\mid D = 0]$ contains a selection-bias term even when $Z$ is randomised.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $D$ is chosen by people (types), not assigned: the treated are compliers-with-offer plus always-takers; the untreated are compliers-without-offer plus never-takers.\n", "\n", "**Step 2.** Apply Day 34's decomposition with $D$ as the treatment: naive $=$ ATT $+ \\{E[Y(0) \\mid D = 1] - E[Y(0) \\mid D = 0]\\}$.\n", "\n", "**Step 3.** The bracket compares $Y(0)$ across different type mixtures, so it is generally not zero.\n", "\n", "**Answer:** as-treated analysis throws away the randomisation and reintroduces selection bias.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · When exclusion fails\n", "\n", "Suppose the offer letter itself raises employment by $\\delta$ for everyone (e.g. it signals that the government is watching). Show the Wald ratio equals CACE $+ \\delta/\\pi_c$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Each unit's outcome difference becomes (its training effect if a complier, else 0) $+ \\delta$.\n", "\n", "**Step 2.** $\\text{ITT}_Y = \\pi_c\\,\\text{CACE} + \\delta$, while $\\text{ITT}_D = \\pi_c$ is unchanged.\n", "\n", "**Step 3.** Ratio: $\\text{CACE} + \\delta/\\pi_c$. $\\blacksquare$\n", "\n", "A small direct effect is magnified when compliance is low: exclusion matters most for weak instruments.\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · A training voucher\n", "\n", "Training vouchers are randomised; 60% of winners enrol and no losers can. Winners earn \\$1,000 more per year than losers. Compute the ITT, the CACE, and say which a finance ministry deciding whether to fund vouchers should use.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\text{ITT}_Y = 1{,}000$; $\\text{ITT}_D = 0.6$.\n", "\n", "**Step 2.** CACE $= 1{,}000/0.6 \\approx 1{,}667$, which equals the ATT here (one-sided non-compliance).\n", "\n", "**Step 3.** The ministry funds *vouchers*, not compulsory training, and 40% of winners will not use them.\n", "\n", "**Answer:** ITT \\$1,000 per voucher (for the cost–benefit of the voucher policy); CACE ≈ \\$1,667 per trainee (for the value of training to those who take it).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Flu-shot encouragement\n", "\n", "Doctors are randomly encouraged by letter to vaccinate elderly patients. Vaccination rises from 20% to 30%; flu-related hospitalisation falls from 9% to 8%. Compute the CACE and state the key assumption.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\text{ITT}_D = 0.30 - 0.20 = 0.10$; $\\text{ITT}_Y = 0.08 - 0.09 = -0.01$.\n", "\n", "**Step 2.** CACE $= -0.01/0.10 = -0.10$.\n", "\n", "**Step 3.** Key assumption: exclusion. The letter must affect hospitalisation only through vaccination, not, for example, by making doctors more attentive in general.\n", "\n", "**Answer:** among patients vaccinated because of the encouragement, vaccination cuts hospitalisation risk by 10 points, if the letter has no other effect.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Adherence in a drug trial\n", "\n", "In a heart-disease trial, patients who took at least 80% of their *placebo* pills had 15% mortality versus 28% for poor adherers. What does this imply for per-protocol analyses of the active drug?\n", "\n", "
Show solution\n", "\n", "**Step 1.** A placebo has no effect, so the 13-point gap is pure selection: good adherers are healthier or more careful.\n", "\n", "**Step 2.** Comparing adherent patients on the drug with all placebo patients (or with non-adherent ones) would credit the drug with this gap.\n", "\n", "**Answer:** per-protocol and as-treated analyses can show large \"effects\" of pills that do nothing; analyse by assignment (ITT) and, if needed, estimate the CACE with assignment as the instrument. (This pattern was famously documented in the Coronary Drug Project.)\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Free bednets\n", "\n", "Free insecticide-treated bednets are offered to randomly chosen households; 65% use them. A health minister asks: \"What will happen to malaria if we offer free nets nationwide?\" Which estimand answers this, and what extra assumption is needed?\n", "\n", "
Show solution\n", "\n", "**Step 1.** The policy is the *offer*, with realistic take-up: the ITT is the answer.\n", "\n", "**Step 2.** Extrapolating nationwide assumes similar take-up and effects elsewhere (external validity), and no change in spillovers (nets protect neighbours, Day 32) at full scale.\n", "\n", "**Answer:** the ITT, provided take-up and spillovers in the study resemble national conditions.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Low take-up\n", "\n", "An encouragement design raises gym attendance by only 2 points; the ITT on blood pressure is −0.1 mmHg (SE 0.2). Compute the CACE and its approximate SE, and comment.\n", "\n", "
Show solution\n", "\n", "**Step 1.** CACE $= -0.1/0.02 = -5$ mmHg.\n", "\n", "**Step 2.** Ignoring uncertainty in $\\text{ITT}_D$, SE $\\approx 0.2/0.02 = 10$ mmHg (delta method, Day 12).\n", "\n", "**Step 3.** 95% CI ≈ $-5 \\pm 19.6$: from −25 to +15 mmHg.\n", "\n", "**Answer:** with such a weak first stage, the complier effect is uninformative, and any small exclusion violation would be magnified 50 times (Exercise 10).\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "rng = np.random.default_rng(41)\n", "n = 400_000\n", "types = rng.choice([\"complier\", \"always\", \"never\"], n, p=[0.5, 0.1, 0.4])\n", "Z = rng.random(n) < 0.5\n", "D = np.where(types == \"always\", 1, np.where(types == \"never\", 0, Z.astype(int)))\n", "base = np.select([types == \"always\", types == \"never\"], [0.55, 0.30], 0.40) # different Y(0) by type\n", "effect = np.where(types == \"complier\", 0.10, 0.05)\n", "Y = rng.random(n) < base + effect * D\n", "itt_y = Y[Z].mean() - Y[~Z].mean(); itt_d = D[Z].mean() - D[~Z].mean()\n", "print(f\"ITT_Y {itt_y:.3f}, ITT_D {itt_d:.3f}, CACE {itt_y / itt_d:.3f} (truth 0.10)\")\n", "print(f\"as-treated {Y[D == 1].mean() - Y[D == 0].mean():.3f}\")\n", "print(f\"per-protocol {Y[Z & (D == 1)].mean() - Y[~Z & (D == 0)].mean():.3f}\")\n", "print(\"flu-shot CACE:\", round((0.08 - 0.09) / (0.30 - 0.20), 3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- Randomising an offer identifies intention-to-treat effects, often the policy-relevant quantity.\n", "- Under exclusion and monotonicity, the effect for compliers is ITT_Y / ITT_D; with one-sided non-compliance it equals the ATT.\n", "- As-treated and per-protocol comparisons discard the randomisation and reintroduce selection bias." ] } ] }