{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 79: Bounds & Partial Identification (Manski, Lee)\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "When the assumptions needed for a point estimate are not credible, report what the data alone can say. Worst-case (Manski) bounds use only the outcome's range, monotonicity assumptions shrink them, and Lee bounds handle outcomes that are missing because of treatment, such as wages observed only for the employed.\n", "\n", "**Learning outcomes**\n", "\n", "- Derive worst-case bounds on mean potential outcomes and the ATE for a bounded outcome.\n", "- Tighten bounds with monotone treatment response and monotone treatment selection.\n", "- Compute Lee bounds by trimming when treatment affects who is observed.\n", "- Report identified sets and their confidence intervals in applied work." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### The missing-data problem\n", "\n", "For a binary treatment $D$, $E[Y(1)] = E[Y \\mid D = 1]P(D = 1) + E[Y(1) \\mid D = 0]P(D = 0)$. The data reveal everything except $E[Y(1) \\mid D = 0]$ (and, symmetrically, $E[Y(0) \\mid D = 1]$).\n", "\n", "### Worst-case (Manski) bounds\n", "\n", "If $Y \\in [y_L, y_U]$, the unobserved term lies in $[y_L, y_U]$, so\n", "\n", "$$\n", "E[Y(1)] \\in \\big[E[Y \\mid D = 1]p + y_L(1 - p),\\;\\; E[Y \\mid D = 1]p + y_U(1 - p)\\big], \\quad p = P(D = 1),\n", "$$\n", "\n", "and similarly for $E[Y(0)]$. The ATE lower bound is the lower bound of $E[Y(1)]$ minus the upper bound of $E[Y(0)]$; the ATE upper bound is the upper bound of $E[Y(1)]$ minus the lower bound of $E[Y(0)]$. Their width is always $y_U - y_L$, and they always contain zero: data alone cannot sign the effect.\n", "\n", "### Monotonicity assumptions (Manski and Pepper)\n", "\n", "- **Monotone treatment response (MTR):** $Y(1) \\ge Y(0)$ for everyone. The ATE is at least 0.\n", "- **Monotone treatment selection (MTS):** those who choose treatment have weakly higher potential outcomes: $E[Y(d) \\mid D = 1] \\ge E[Y(d) \\mid D = 0]$. Then the naive difference in means is an **upper** bound on the ATE.\n", "- Together: $0 \\le ATE \\le E[Y \\mid D = 1] - E[Y \\mid D = 0]$.\n", "\n", "### Lee bounds for selective attrition\n", "\n", "In a randomised experiment, the outcome (wages) is observed only for some units (the employed), and treatment changes who is observed. Assume **monotonicity of selection**: treatment can only make units observed, never unobserved. Then observed treated units contain all \"always-observed\" units plus some units observed only because of treatment, a share\n", "\n", "$$\n", "p = \\frac{q_1 - q_0}{q_1}\n", "$$\n", "\n", "of treated respondents, where $q_d$ is the response rate in arm $d$. Trimming the top $p$ (or bottom $p$) of the treated respondents' outcome distribution gives the lowest (or highest) possible mean for always-observed units; subtracting the control respondents' mean bounds the effect for the always-observed.\n", "\n", "### Inference\n", "\n", "Sample bounds are estimates. For a 95% confidence interval covering the true parameter (not the whole set), Imbens and Manski use $[\\hat L - c\\,\\operatorname{SE}_L,\\; \\hat U + c\\,\\operatorname{SE}_U]$ with $c$ between 1.645 (wide sets) and 1.96 (narrow sets).\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Worst-case bounds\n", "\n", "Binary outcome. $P(D = 1) = 0.4$, $P(Y = 1 \\mid D = 1) = 0.7$, $P(Y = 1 \\mid D = 0) = 0.5$. Bound $E[Y(1)]$, $E[Y(0)]$ and the ATE.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $E[Y(1)] \\in [0.7(0.4) + 0,\\; 0.28 + 0.6] = [0.28, 0.88]$.\n", "\n", "**Step 2.** $E[Y(0)] \\in [0.5(0.6) + 0,\\; 0.30 + 0.4] = [0.30, 0.70]$.\n", "\n", "**Step 3.** ATE $\\in [0.28 - 0.70,\\; 0.88 - 0.30] = [-0.42, 0.58]$.\n", "\n", "**Answer:** width 1, containing zero.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Width\n", "\n", "Check that the width in Exercise 1 equals $y_U - y_L$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $0.58 - (-0.42) = 1.00 = 1 - 0$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · Adding MTR\n", "\n", "Under MTR, what are the ATE bounds in Exercise 1?\n", "\n", "
Show solution\n", "\n", "**Step 1.** MTR implies $ATE \\ge 0$, and it does not lower the upper bound here.\n", "\n", "**Answer:** $[0, 0.58]$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · Adding MTS\n", "\n", "Under MTS (and MTR), what are the bounds?\n", "\n", "
Show solution\n", "\n", "**Step 1.** MTS: ATE ≤ naive difference $0.7 - 0.5 = 0.2$.\n", "\n", "**Answer:** $[0, 0.2]$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Lee's trimming share\n", "\n", "Response rates are 0.9 (treated) and 0.8 (control). What share of treated respondents is trimmed?\n", "\n", "
Show solution\n", "\n", "**Step 1.** $p = (0.9 - 0.8)/0.9 \\approx 0.111$.\n", "\n", "**Answer:** about 11% of treated respondents.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · Worst-case bounds\n", "\n", "Prove the bounds on $E[Y(1)]$ and that they are sharp (attainable).\n", "\n", "
Show solution\n", "\n", "**Step 1.** Law of total expectation: $E[Y(1)] = E[Y \\mid D = 1]p + E[Y(1) \\mid D = 0](1 - p)$, using consistency for the treated.\n", "\n", "**Step 2.** $E[Y(1) \\mid D = 0] \\in [y_L, y_U]$ since $Y(1)$ is bounded.\n", "\n", "**Step 3.** Substitute the extremes. Each value in the interval is attained by a distribution of $Y(1)$ among the untreated that is consistent with the data, so the bounds are sharp. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · The width is always the range\n", "\n", "Prove that the width of the ATE bounds is $y_U - y_L$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Width of the $E[Y(1)]$ bounds: $(y_U - y_L)(1 - p)$; of $E[Y(0)]$: $(y_U - y_L)p$.\n", "\n", "**Step 2.** ATE width = sum of the two widths $= (y_U - y_L)(1 - p + p)$. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · MTS gives an upper bound\n", "\n", "Prove that under MTS, $ATE \\le E[Y \\mid D = 1] - E[Y \\mid D = 0]$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** MTS for $d = 1$: $E[Y(1) \\mid D = 0] \\le E[Y(1) \\mid D = 1] = E[Y \\mid D = 1]$, so $E[Y(1)] \\le E[Y \\mid D = 1]$.\n", "\n", "**Step 2.** MTS for $d = 0$: $E[Y(0) \\mid D = 1] \\ge E[Y(0) \\mid D = 0] = E[Y \\mid D = 0]$, so $E[Y(0)] \\ge E[Y \\mid D = 0]$.\n", "\n", "**Step 3.** Subtract. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Lee bounds\n", "\n", "Assume random assignment and monotone selection ($S(1) \\ge S(0)$). Show that treated respondents are a mixture of always-observed units (share $1 - p$) and units observed only if treated (share $p$), and that trimming the top $p$ gives a lower bound for the always-observed treated mean.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Control respondents have $S(0) = 1$: always-observed only (by monotonicity). Their share of the population is $q_0$.\n", "\n", "**Step 2.** Treated respondents have $S(1) = 1$: always-observed ($q_0$) plus marginal units ($q_1 - q_0$); the marginal share among them is $p$.\n", "\n", "**Step 3.** The always-observed treated outcomes are an unknown $(1 - p)$ part of the treated respondents' distribution; the smallest possible mean of such a part is the mean of the lowest $1 - p$ (trim the top $p$), the largest is the mean after trimming the bottom $p$.\n", "\n", "**Step 4.** Control respondents' mean is the always-observed untreated mean. Subtract. $\\blacksquare$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Covering the parameter, not the set\n", "\n", "Explain why $[\\hat L - 1.645\\,\\operatorname{SE},\\; \\hat U + 1.645\\,\\operatorname{SE}]$ covers the true parameter with probability at least about 95% when the identified set is wide.\n", "\n", "
Show solution\n", "\n", "**Step 1.** The parameter lies in $[L, U]$. If it is near $L$, only the lower end can miss, which happens with probability about 5% using the one-sided 1.645.\n", "\n", "**Step 2.** When the set is wide, the parameter cannot be near both ends at once, so at most one tail matters.\n", "\n", "**Answer:** coverage of the parameter is about 95%; for narrow sets the critical value rises towards 1.96 (Imbens–Manski). $\\blacksquare$\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Job training and wages\n", "\n", "In a randomised training programme, 90% of the treatment group and 80% of the control group are employed (wages observed). Treated workers' wages are uniform on \\$0–\\$900 a week; control workers' mean wage is \\$380. Compute Lee bounds on the wage effect for always-employed workers.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Trim share $p = 0.111$; with a uniform on [0, 900], that is the top or bottom \\$100.\n", "\n", "**Step 2.** Trim the top: mean of [0, 800] = \\$400. Trim the bottom: mean of [100, 900] = \\$500.\n", "\n", "**Step 3.** Bounds: $[400 - 380,\\; 500 - 380] = [20, 120]$.\n", "\n", "**Answer:** the programme raised weekly wages of always-employed workers by \\$20–\\$120; the naive comparison (\\$450 − \\$380 = \\$70) mixes in newly employed workers.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Smoking and a health outcome\n", "\n", "In a survey, 25% smoke; 30% of smokers and 15% of non-smokers report a chronic respiratory condition. Give worst-case bounds on the effect of smoking on the condition, then bounds under MTR and MTS.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $E[Y(1)] \\in [0.075, 0.825]$; $E[Y(0)] \\in [0.1125, 0.3625]$.\n", "\n", "**Step 2.** Worst case: $[0.075 - 0.3625,\\; 0.825 - 0.1125] = [-0.2875, 0.7125]$.\n", "\n", "**Step 3.** MTR (smoking never improves respiratory health) and MTS (smokers would have worse outcomes either way): $[0, 0.30 - 0.15] = [0, 0.15]$.\n", "\n", "**Answer:** data alone: anything from −29 to +71 points; with two plausible monotonicity assumptions: between 0 and 15 points.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Returns to schooling under MTR and MTS\n", "\n", "Mean log wages are 2.9 for college graduates and 2.5 for others. Under MTR (schooling never lowers wages) and MTS (people who chose college would earn more at any schooling level), bound the effect of college on log wages.\n", "\n", "
Show solution\n", "\n", "**Step 1.** MTR: effect ≥ 0.\n", "\n", "**Step 2.** MTS: effect ≤ naive difference $2.9 - 2.5 = 0.4$.\n", "\n", "**Answer:** between 0 and 0.4 log points; Manski and Pepper report bounds of this kind as a transparent alternative to IV.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Missing outcomes in a vaccine trial\n", "\n", "In a trial with 10,000 per arm, 50 infections are observed among 9,500 followed-up vaccinated participants and 200 among 9,600 placebo participants. Compute worst-case bounds on the infection risk in each arm, treating lost participants as all infected or all uninfected.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Vaccine arm: $[50/10{,}000,\\; (50 + 500)/10{,}000] = [0.005, 0.055]$.\n", "\n", "**Step 2.** Placebo: $[200/10{,}000,\\; (200 + 400)/10{,}000] = [0.020, 0.060]$.\n", "\n", "**Step 3.** Risk difference bounds: $[0.005 - 0.060,\\; 0.055 - 0.020] = [-0.055, 0.035]$.\n", "\n", "**Answer:** worst-case bounds cannot even sign the effect, because attrition (5% and 4%) is large relative to infection rates; reducing loss to follow-up matters as much as sample size.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · When to report bounds\n", "\n", "A ministry asks for the effect of a microcredit programme with voluntary take-up and no instrument. What should be delivered?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Worst-case bounds (usually uninformative, but honest).\n", "\n", "**Step 2.** Bounds under explicit, defensible assumptions (MTR if credit cannot reduce profits on average; MTS if more entrepreneurial households select in), showing how each assumption narrows the set.\n", "\n", "**Step 3.** Point estimates under stronger assumptions (selection on observables) alongside, with sensitivity analysis (Day 95).\n", "\n", "**Answer:** a ladder from weak to strong assumptions, so the reader sees what each conclusion rests on.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "def manski(p, m1, m0, lo=0, hi=1):\n", " y1 = (m1 * p + lo * (1 - p), m1 * p + hi * (1 - p)); y0 = (m0 * (1 - p) + lo * p, m0 * (1 - p) + hi * p)\n", " return np.round(y1, 4), np.round(y0, 4), np.round((y1[0] - y0[1], y1[1] - y0[0]), 4)\n", "for name, args in [(\"Ex 1\", (.4, .7, .5)), (\"Ex 12\", (.25, .30, .15))]:\n", " y1, y0, ate = manski(*args); print(f\"{name}: E[Y(1)] {y1.tolist()}, E[Y(0)] {y0.tolist()}, ATE {ate.tolist()}\")\n", "print(\"Ex 14:\", np.round([50 / 1e4 - 600 / 1e4, 550 / 1e4 - 200 / 1e4], 4).tolist())\n", "\n", "# Lee bounds: Exercise 11 by formula, then a simulated experiment\n", "p = (.9 - .8) / .9\n", "print(f\"Ex 11: trim share {p:.3f}, bounds [{900 * (1 - p) / 2 - 380:.0f}, {(900 * p + 900) / 2 - 380:.0f}]\")\n", "rng = np.random.default_rng(79); n = 400_000\n", "Z = rng.random(n) < .5\n", "t = rng.choice([0, 1, 2], n, p=[.8, .1, .1]) # always employed, employed only if treated, never\n", "w0 = rng.lognormal(6, .5, n)\n", "w = np.where(Z, 1.1 * w0, w0); w = np.where(t == 1, .7 * w, w)\n", "S = (t == 0) | ((t == 1) & Z)\n", "q1, q0 = S[Z].mean(), S[~Z].mean(); p = (q1 - q0) / q1\n", "wt = np.sort(w[Z & S]); k = int(round(p * len(wt))); c = w[~Z & S].mean()\n", "print(f\"trim {p:.3f}; Lee bounds [{wt[:len(wt) - k].mean() - c:.1f}, {wt[k:].mean() - c:.1f}]; \"\n", " f\"truth for always-employed {(.1 * w0[t == 0]).mean():.1f}; naive {w[Z & S].mean() - c:.1f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- With a bounded outcome and no assumptions, the ATE is bounded by an interval of width $y_U - y_L$ that contains zero.\n", "- Monotone response and selection assumptions narrow the bounds transparently; MTS makes the naive difference an upper bound.\n", "- Lee bounds trim the treated respondents' distribution to handle outcomes observed only for some units, assuming treatment only adds respondents." ] } ] }