{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 4: Common Distributions & the Data-Generating Process\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "The handful of distributions that model most economic data, and how to write a DGP with potential outcomes.\n", "\n", "**Learning outcomes**\n", "\n", "- Compute probabilities, means and variances for the Bernoulli, binomial, Poisson, uniform, normal, exponential and log-normal distributions.\n", "- Derive the mean of the binomial, Poisson and log-normal distributions.\n", "- Write a data-generating process (DGP) for a randomised and for a confounded study.\n", "- Show how a confounded DGP creates selection bias in a naive comparison." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### A small toolkit of distributions\n", "\n", "| Distribution | Typical use | Support | Mean | Variance |\n", "|---|---|---|---|---|\n", "| Bernoulli($p$) | treated or not, employed or not | $\\{0, 1\\}$ | $p$ | $p(1-p)$ |\n", "| Binomial($n, p$) | successes in $n$ trials | $\\{0, \\dots, n\\}$ | $np$ | $np(1-p)$ |\n", "| Poisson($\\lambda$) | counts: patents, visits, claims | $\\{0, 1, \\dots\\}$ | $\\lambda$ | $\\lambda$ |\n", "| Uniform($a, b$) | random draws, lotteries | $[a, b]$ | $\\tfrac{a+b}{2}$ | $\\tfrac{(b-a)^2}{12}$ |\n", "| Normal($\\mu, \\sigma^2$) | test scores, errors, estimators | $\\mathbb{R}$ | $\\mu$ | $\\sigma^2$ |\n", "| Exponential($\\lambda$) | durations: unemployment spells | $[0, \\infty)$ | $1/\\lambda$ | $1/\\lambda^2$ |\n", "| Log-normal($\\mu, \\sigma^2$) | wages, firm size, health costs | $(0, \\infty)$ | $e^{\\mu + \\sigma^2/2}$ | $(e^{\\sigma^2} - 1)e^{2\\mu + \\sigma^2}$ |\n", "\n", "Key formulas:\n", "\n", "$$\n", "P(X = k) = \\binom{n}{k}p^k(1-p)^{n-k}, \\qquad P(X = k) = \\frac{e^{-\\lambda}\\lambda^k}{k!}, \\qquad f(x) = \\frac{1}{\\sigma\\sqrt{2\\pi}}e^{-\\frac{(x-\\mu)^2}{2\\sigma^2}}.\n", "$$\n", "\n", "Any normal variable can be **standardised**: $Z = (X - \\mu)/\\sigma \\sim N(0, 1)$, and $\\Phi$ denotes the standard normal CDF. Independent normals add: if $X \\sim N(\\mu_1, \\sigma_1^2)$ and $Y \\sim N(\\mu_2, \\sigma_2^2)$ are independent, $X + Y \\sim N(\\mu_1 + \\mu_2, \\sigma_1^2 + \\sigma_2^2)$.\n", "\n", "### The data-generating process\n", "\n", "A **DGP** is a recipe that says how each variable is produced. Writing one forces you to state your assumptions, and simulating it lets you check whether an estimator recovers the truth. A causal DGP specifies potential outcomes and how treatment is assigned:\n", "\n", "$$\n", "\\begin{aligned}\n", "U &\\sim \\text{Bernoulli}(0.5) && \\text{(unobserved ability)}\\\\\n", "T \\mid U &\\sim \\text{Bernoulli}(0.2 + 0.6U) && \\text{(who takes the programme)}\\\\\n", "Y(t) &= 10 + 5t + 20U + \\varepsilon, \\quad \\varepsilon \\sim N(0, 1) && \\text{(potential outcomes)}\\\\\n", "Y &= Y(T) && \\text{(what we observe)}\n", "\\end{aligned}\n", "$$\n", "\n", "Here the true effect is 5 for everyone. Whether a naive comparison recovers 5 depends on the second line (Exercise 15).\n", "\n", "> **Causal link.** Every method in this course will be checked against a simulated DGP whose true effect you know. Learning to write DGPs today is learning how to test causal estimators.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Binomial attendance\n", "\n", "Each of 10 training sessions is attended with probability 0.3, independently. Find $P(X = 3)$, $E[X]$ and $\\operatorname{Var}(X)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $X \\sim \\text{Binomial}(n = 10, p = 0.3)$. The number of ways to pick which 3 sessions are attended: $\\binom{10}{3} = 120$.\n", "\n", "**Step 2.** Probability of one particular pattern with 3 attended and 7 missed: $0.3^3 \\times 0.7^7 = 0.027 \\times 0.08235 \\approx 0.002224$.\n", "\n", "**Step 3.** Multiply: $P(X = 3) = 120 \\times 0.002224 \\approx 0.2668$.\n", "\n", "**Step 4.** Moments: $E[X] = np = 3$ and $\\operatorname{Var}(X) = np(1 - p) = 10(0.3)(0.7) = 2.1$.\n", "\n", "**Answer:** $P(X = 3) \\approx 0.267$, mean 3 sessions, variance 2.1.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Poisson claims\n", "\n", "Insurance claims per month are Poisson with $\\lambda = 4$. Find $P(X = 0)$ and $P(X \\le 1)$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Poisson pmf: $P(X = k) = e^{-\\lambda}\\lambda^k/k!$ with $\\lambda = 4$.\n", "\n", "**Step 2.** $P(X = 0) = e^{-4} \\approx 0.0183$.\n", "\n", "**Step 3.** $P(X = 1) = e^{-4}\\cdot 4 \\approx 0.0733$.\n", "\n", "**Step 4.** $P(X \\le 1) = 0.0183 + 0.0733 = 5e^{-4} \\approx 0.0916$.\n", "\n", "**Answer:** a claim-free month has probability 1.8%; at most one claim, 9.2%.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · Standardising wages\n", "\n", "Wages are $N(50{,}000,\\; 10{,}000^2)$. What share earn more than \\$65,000?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Standardise: $z = (65{,}000 - 50{,}000)/10{,}000 = 1.5$.\n", "\n", "**Step 2.** $P(W > 65{,}000) = P(Z > 1.5) = 1 - \\Phi(1.5) = 1 - 0.9332 = 0.0668$.\n", "\n", "**Answer:** about 6.7% of workers earn more than \\$65,000.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · Unemployment duration\n", "\n", "Spell length (months) is exponential with mean 2, i.e. $\\lambda = 0.5$. What is $P(T > 3)$?\n", "\n", "
Show solution\n", "\n", "For the exponential, $P(T > t) = e^{-\\lambda t}$, so $P(T > 3) = e^{-1.5} \\approx 0.223$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · A uniform lottery\n", "\n", "A lottery assigns a voucher uniformly between \\$20 and \\$40. Find the mean and variance.\n", "\n", "
Show solution\n", "\n", "Mean $(20 + 40)/2 = 30$; variance $(40 - 20)^2/12 = 400/12 \\approx 33.3$.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · Binomial moments from Bernoullis\n", "\n", "Write $X = \\sum_{i=1}^n B_i$ with independent $B_i \\sim \\text{Bernoulli}(p)$. Derive $E[X]$ and $\\operatorname{Var}(X)$.\n", "\n", "
Show solution\n", "\n", "By linearity, $E[X] = \\sum_i E[B_i] = np$. Because the $B_i$ are independent, their covariances are zero (Day 5), so variances add: $\\operatorname{Var}(X) = \\sum_i p(1-p) = np(1-p)$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · The Poisson mean\n", "\n", "Show that $E[X] = \\lambda$ for $X \\sim \\text{Poisson}(\\lambda)$.\n", "\n", "
Show solution\n", "\n", "$$\n", "E[X] = \\sum_{k=0}^\\infty k\\frac{e^{-\\lambda}\\lambda^k}{k!} = \\lambda e^{-\\lambda}\\sum_{k=1}^\\infty \\frac{\\lambda^{k-1}}{(k-1)!} = \\lambda e^{-\\lambda}\\sum_{j=0}^\\infty \\frac{\\lambda^{j}}{j!} = \\lambda e^{-\\lambda}e^{\\lambda} = \\lambda.\n", "$$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Memorylessness\n", "\n", "Prove that the exponential distribution satisfies $P(T > s + t \\mid T > s) = P(T > t)$.\n", "\n", "
Show solution\n", "\n", "$$\n", "P(T > s + t \\mid T > s) = \\frac{P(T > s + t)}{P(T > s)} = \\frac{e^{-\\lambda(s+t)}}{e^{-\\lambda s}} = e^{-\\lambda t} = P(T > t).\n", "$$\n", "\n", "A worker unemployed for 6 months is no more likely to find a job next month than one just laid off. Real data usually show *duration dependence*, which is evidence against the exponential model.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Sums and differences of normals\n", "\n", "$X \\sim N(1, 4)$ and $Y \\sim N(2, 9)$ are independent. Find the distributions of $X + Y$ and $X - Y$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $X + Y \\sim N(3, 13)$.\n", "\n", "**Step 2.** For the difference, $\\operatorname{Var}(X - Y) = \\operatorname{Var}(X) + (-1)^2\\operatorname{Var}(Y) = 13$, so $X - Y \\sim N(-1, 13)$.\n", "\n", "**Step 3.** Variances add even when you subtract, which is why the standard error of a difference in means is larger than either group's.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · The log-normal mean\n", "\n", "If $\\ln Y \\sim N(\\mu, \\sigma^2)$, show that $E[Y] = e^{\\mu + \\sigma^2/2}$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Let $X = \\ln Y$.\n", "\n", "**Step 2.** Then $E[Y] = E[e^X] = \\int e^x \\frac{1}{\\sigma\\sqrt{2\\pi}}e^{-(x-\\mu)^2/(2\\sigma^2)}dx$.\n", "\n", "**Step 3.** Complete the square in the exponent:\n", "\n", "$$\n", "x - \\frac{(x-\\mu)^2}{2\\sigma^2} = -\\frac{\\big(x - (\\mu + \\sigma^2)\\big)^2}{2\\sigma^2} + \\mu + \\frac{\\sigma^2}{2}.\n", "$$\n", "\n", "**Step 4.** The remaining integral is a $N(\\mu + \\sigma^2, \\sigma^2)$ density, which integrates to 1, leaving $E[Y] = e^{\\mu + \\sigma^2/2}$.\n", "\n", "**Step 5.** Note $E[Y] > e^{E[\\ln Y]} = e^{\\mu}$, as Jensen requires.\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Rare events\n", "\n", "1,000 firms each go bankrupt this year with probability 0.002, independently. Compare the exact binomial probability of no bankruptcies with the Poisson approximation.\n", "\n", "
Show solution\n", "\n", "**Step 1 (exact).** $X \\sim \\text{Binomial}(1000, 0.002)$, so $P(X = 0) = 0.998^{1000}$.\n", "\n", "**Step 2.** Evaluate with logs: $1000\\ln 0.998 = 1000 \\times (-0.002002) = -2.002$, so $P(X = 0) = e^{-2.002} \\approx 0.1350$.\n", "\n", "**Step 3 (approximation).** $\\lambda = np = 1000 \\times 0.002 = 2$, so the Poisson gives $P(X = 0) = e^{-2} \\approx 0.1353$.\n", "\n", "**Answer:** 0.1350 versus 0.1353. With many firms and a small probability each, the Poisson is an excellent approximation.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Overdispersion\n", "\n", "Doctor visits have sample mean 3 and sample variance 12. Is a Poisson model reasonable?\n", "\n", "
Show solution\n", "\n", "**Step 1.** A Poisson model forces $\\operatorname{Var}(X) = E[X]$.\n", "\n", "**Step 2.** Here the variance is $12/3 = 4$ times the mean: **overdispersion**.\n", "\n", "**Step 3.** Likely cause: unobserved differences between people (health, insurance, distance to the clinic) make the visit rate vary across individuals, which adds variance.\n", "\n", "**Answer:** a pure Poisson model is not reasonable. Use a negative binomial model, or Poisson regression with robust standard errors.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Mean versus median wage\n", "\n", "Log wages are $N(10, 0.5^2)$. Find the median and mean wage.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\ln Y$ is normal, so its median equals its mean, 10. Because $\\exp$ is increasing, it maps the median of $\\ln Y$ to the median of $Y$:\n", "\n", "$$\n", "\\text{median}(Y) = e^{10} \\approx 22{,}026.\n", "$$\n", "\n", "**Step 2.** Mean of a log-normal (Exercise 10) with $\\mu = 10$, $\\sigma^2 = 0.25$:\n", "\n", "$$\n", "E[Y] = e^{10 + 0.25/2} = e^{10.125} \\approx 24{,}959.\n", "$$\n", "\n", "**Answer:** median ≈ \\$22,026 and mean ≈ \\$24,959. Right-skewed variables like wages have a mean above the median.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · A randomised tutoring programme\n", "\n", "A school district simulates a tutoring RCT. A pupil's test score without tutoring is $Y(0) \\sim N(50, 10^2)$; tutoring adds 5 points to everyone, $Y(1) = Y(0) + 5$; and a coin flip assigns tutoring, $T \\sim \\text{Bernoulli}(0.5)$, independently of $Y(0)$. Find $E[Y \\mid T = 1] - E[Y \\mid T = 0]$ and $\\operatorname{Var}(Y)$ for the observed score $Y$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Observed score: $Y = Y(0) + 5T$.\n", "\n", "**Step 2.** Because $T$ is independent of $Y(0)$, $E[Y(0) \\mid T = t] = 50$ for both arms. So\n", "\n", "$$\n", "E[Y \\mid T = 1] = 50 + 5 = 55, \\qquad E[Y \\mid T = 0] = 50.\n", "$$\n", "\n", "**Step 3.** The difference is $55 - 50 = 5$, the true effect.\n", "\n", "**Step 4.** Variance of a sum of independent pieces: $\\operatorname{Var}(Y) = \\operatorname{Var}(Y(0)) + 5^2\\operatorname{Var}(T) = 100 + 25(0.25) = 106.25$.\n", "\n", "**Answer:** the difference in means is 5 (unbiased for the effect), and $\\operatorname{Var}(Y) = 106.25$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · A confounded job-training DGP\n", "\n", "Use the DGP in the theory block, read as job training: $U$ is high motivation ($U \\sim \\text{Bern}(0.5)$), motivated workers enrol more often, $P(T = 1 \\mid U) = 0.2 + 0.6U$, and earnings (in \\$1,000) are $Y = 10 + 5T + 20U + \\varepsilon$. Compute $E[Y \\mid T = 1] - E[Y \\mid T = 0]$ and the selection bias.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Share enrolled: $P(T = 1) = 0.5(0.8) + 0.5(0.2) = 0.5$.\n", "\n", "**Step 2.** Bayes' theorem for motivation among trainees and non-trainees:\n", "\n", "$$\n", "P(U = 1 \\mid T = 1) = \\frac{0.8 \\times 0.5}{0.5} = 0.8, \\qquad P(U = 1 \\mid T = 0) = \\frac{0.2 \\times 0.5}{0.5} = 0.2.\n", "$$\n", "\n", "**Step 3.** Conditional means ($E[\\varepsilon] = 0$):\n", "\n", "$$\n", "E[Y \\mid T = 1] = 10 + 5 + 20(0.8) = 31, \\qquad E[Y \\mid T = 0] = 10 + 20(0.2) = 14.\n", "$$\n", "\n", "**Step 4.** Naive difference: $31 - 14 = 17$. True effect: 5. Selection bias: $17 - 5 = 12$, which is $20 \\times (0.8 - 0.2)$.\n", "\n", "**Answer:** the naive comparison says training is worth \\$17k; \\$12k of that is motivated workers selecting into training.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "from scipy import stats\n", "\n", "print(\"Binomial P(X=3):\", round(stats.binom.pmf(3, 10, 0.3), 4))\n", "print(\"Poisson P(X<=1):\", round(stats.poisson.cdf(1, 4), 4))\n", "print(\"Normal tail:\", round(stats.norm.sf(65_000, 50_000, 10_000), 4))\n", "print(\"Exact vs Poisson:\", round(0.998**1000, 4), round(np.exp(-2), 4))\n", "print(\"Log-normal mean:\", round(np.exp(10 + 0.5**2 / 2)))\n", "\n", "# Exercise 15: simulate the confounded DGP\n", "rng = np.random.default_rng(42)\n", "n = 1_000_000\n", "U = rng.binomial(1, 0.5, n)\n", "T = rng.binomial(1, 0.2 + 0.6 * U)\n", "Y = 10 + 5 * T + 20 * U + rng.normal(0, 1, n)\n", "print(\"naive difference:\", round(Y[T == 1].mean() - Y[T == 0].mean(), 2), \"(true effect 5)\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- A few distributions cover most data: Bernoulli/binomial for binary, Poisson for counts, exponential for durations, normal and log-normal for continuous outcomes.\n", "- A DGP makes assumptions explicit and gives you a ground truth to test estimators against.\n", "- When an unobserved variable drives both treatment and outcome, a naive comparison mixes the effect with selection bias." ] } ] }