{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 1: Probability Axioms & Set Theory\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "Sample spaces, events and Kolmogorov's three axioms, and the rules that follow from them.\n", "\n", "**Learning outcomes**\n", "\n", "- Write the sample space and events for a treatment-assignment experiment.\n", "- State Kolmogorov's three axioms and derive the complement, monotonicity and inclusion–exclusion rules from them.\n", "- Bound an unknown joint probability using only the marginals (Boole and Fréchet bounds).\n", "- Count the assignments in a completely randomised experiment." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### Why start here?\n", "\n", "The **fundamental problem of causal inference** is that we never see the same unit both treated and untreated. Every causal estimate is therefore a statement about *populations*, and populations are described with probability. Before we can define an average treatment effect we need to be exact about what \"probability\" means.\n", "\n", "### Sample space, events and set operations\n", "\n", "- The **sample space** $\\Omega$ is the set of all possible outcomes of a random experiment.\n", "- An **event** $A$ is a subset of $\\Omega$. The collection of events we are allowed to measure is written $\\mathcal{F}$.\n", "- For events $A$ and $B$:\n", " - **union** $A \\cup B$: \"$A$ or $B$ (or both)\"\n", " - **intersection** $A \\cap B$: \"both $A$ and $B$\"\n", " - **complement** $A^c = \\Omega \\setminus A$: \"not $A$\"\n", " - $A$ and $B$ are **mutually exclusive** (disjoint) if $A \\cap B = \\emptyset$.\n", "- **De Morgan's laws:** $(A \\cup B)^c = A^c \\cap B^c$ and $(A \\cap B)^c = A^c \\cup B^c$.\n", "\n", "### Kolmogorov's axioms\n", "\n", "A probability measure $P$ assigns a number to each event so that:\n", "\n", "1. **Non-negativity:** $P(A) \\ge 0$ for every event $A$.\n", "2. **Normalisation:** $P(\\Omega) = 1$.\n", "3. **Countable additivity:** if $A_1, A_2, \\dots$ are pairwise disjoint, then\n", "\n", "$$\n", "P\\left(\\bigcup_{i=1}^{\\infty} A_i\\right) = \\sum_{i=1}^{\\infty} P(A_i).\n", "$$\n", "\n", "Everything else in probability is derived from these three lines. Today you will derive the most useful consequences yourself:\n", "\n", "| Rule | Statement |\n", "|---|---|\n", "| Empty set | $P(\\emptyset) = 0$ |\n", "| Complement | $P(A^c) = 1 - P(A)$ |\n", "| Monotonicity | $A \\subseteq B \\Rightarrow P(A) \\le P(B)$ |\n", "| Inclusion–exclusion | $P(A \\cup B) = P(A) + P(B) - P(A \\cap B)$ |\n", "| Boole's inequality | $P(A \\cup B) \\le P(A) + P(B)$ |\n", "| Fréchet bounds | $\\max\\{0,\\, P(A)+P(B)-1\\} \\le P(A \\cap B) \\le \\min\\{P(A), P(B)\\}$ |\n", "\n", "### Equally likely outcomes and counting\n", "\n", "When $\\Omega$ is finite and every outcome is equally likely, $P(A) = |A| / |\\Omega|$. In a **completely randomised experiment** with $n$ units of which exactly $n_1$ are treated, every one of the $\\binom{n}{n_1}$ possible assignments has the same probability. This is the probability space behind Fisher's randomisation test (Phase 3).\n", "\n", "> **Causal link.** The Fréchet bounds are your first taste of *partial identification*: when the data pin down only the marginals, the joint is known only up to an interval. Manski's bounds on treatment effects (Phase 6) use exactly this logic.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · Defining the sample space\n", "\n", "An environmental regulation gives each firm three mutually exclusive options: adopt clean technology ($C$), pay a carbon tax ($T$), or exit the market ($E$). Write the sample space $\\Omega$.\n", "\n", "
Show solution\n", "\n", "The sample space lists every basic outcome:\n", "\n", "$$\n", "\\Omega = \\{C, T, E\\}.\n", "$$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Identifying an event\n", "\n", "Using Exercise 1, write the event $A$ = \"the firm stays in the market\", and its complement.\n", "\n", "
Show solution\n", "\n", "**Step 1.** The firm stays in the market if it adopts clean technology or pays the tax:\n", "\n", "$$\n", "A = \\{C, T\\}.\n", "$$\n", "\n", "**Step 2.** The complement collects every other outcome in $\\Omega$:\n", "\n", "$$\n", "A^c = \\Omega \\setminus A = \\{E\\}.\n", "$$\n", "\n", "**Answer:** $A = \\{C, T\\}$ and $A^c = \\{E\\}$ (the firm exits).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · Normalisation\n", "\n", "If $P(C) = 0.4$ and $P(E) = 0.1$, what is $P(T)$?\n", "\n", "
Show solution\n", "\n", "The three outcomes are disjoint and exhaustive, so by Axioms 2 and 3:\n", "\n", "$$\n", "P(C) + P(T) + P(E) = 1 \\;\\Rightarrow\\; P(T) = 1 - 0.4 - 0.1 = 0.5.\n", "$$\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · A tiny randomised trial\n", "\n", "Two patients are each assigned to treatment ($T$) or control ($C$) by independent fair coin flips. Write $\\Omega$, the event \"exactly one patient is treated\", and its probability.\n", "\n", "
Show solution\n", "\n", "Writing patient 1's arm first: $\\Omega = \\{TT, TC, CT, CC\\}$, four equally likely outcomes.\n", "\n", "\"Exactly one treated\" is $\\{TC, CT\\}$, so its probability is $2/4 = 0.5$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Additivity\n", "\n", "Trial participants have no side effects ($S_0$), mild ($S_1$) or severe ($S_2$) side effects. If $P(S_1) = 0.20$ and $P(S_2) = 0.05$, what is the probability of any side effect? Of none?\n", "\n", "
Show solution\n", "\n", "$S_1$ and $S_2$ are disjoint, so by Axiom 3:\n", "\n", "$$\n", "P(S_1 \\cup S_2) = 0.20 + 0.05 = 0.25.\n", "$$\n", "\n", "\"No side effect\" is the complement: $P(S_0) = 1 - 0.25 = 0.75$.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · The empty set\n", "\n", "Using only the axioms, prove $P(\\emptyset) = 0$.\n", "\n", "
Show solution\n", "\n", "1. $\\Omega$ and $\\emptyset$ are disjoint, and $\\Omega \\cup \\emptyset = \\Omega$.\n", "2. By Axiom 3, $P(\\Omega) = P(\\Omega \\cup \\emptyset) = P(\\Omega) + P(\\emptyset)$.\n", "3. Subtract $P(\\Omega)$ (which is finite, equal to 1) from both sides: $P(\\emptyset) = 0$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · The complement rule\n", "\n", "Prove that $P(A^c) = 1 - P(A)$.\n", "\n", "
Show solution\n", "\n", "1. $A \\cap A^c = \\emptyset$ and $A \\cup A^c = \\Omega$.\n", "2. Axiom 3 gives $P(A) + P(A^c) = P(\\Omega)$.\n", "3. Axiom 2 gives $P(\\Omega) = 1$, so $P(A^c) = 1 - P(A)$.\n", "\n", "A corollary: since $P(A^c) \\ge 0$ (Axiom 1), $P(A) \\le 1$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · Monotonicity\n", "\n", "Prove that if $A \\subseteq B$, then $P(A) \\le P(B)$.\n", "\n", "
Show solution\n", "\n", "1. Write $B = A \\cup (B \\cap A^c)$. The two pieces are disjoint.\n", "2. By Axiom 3, $P(B) = P(A) + P(B \\cap A^c)$.\n", "3. By Axiom 1, $P(B \\cap A^c) \\ge 0$, so $P(B) \\ge P(A)$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Inclusion–exclusion\n", "\n", "Prove $P(A \\cup B) = P(A) + P(B) - P(A \\cap B)$.\n", "\n", "
Show solution\n", "\n", "1. Split the union into disjoint pieces: $A \\cup B = A \\cup (B \\cap A^c)$, so $P(A \\cup B) = P(A) + P(B \\cap A^c)$.\n", "2. Split $B$ into disjoint pieces: $B = (B \\cap A) \\cup (B \\cap A^c)$, so $P(B \\cap A^c) = P(B) - P(A \\cap B)$.\n", "3. Substitute step 2 into step 1:\n", "\n", "$$\n", "P(A \\cup B) = P(A) + P(B) - P(A \\cap B).\n", "$$\n", "\n", "Boole's inequality follows immediately, because $P(A \\cap B) \\ge 0$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · The Fréchet lower bound\n", "\n", "Prove that $P(A \\cap B) \\ge P(A) + P(B) - 1$ (Bonferroni's inequality).\n", "\n", "
Show solution\n", "\n", "1. Rearrange inclusion–exclusion: $P(A \\cap B) = P(A) + P(B) - P(A \\cup B)$.\n", "2. $P(A \\cup B) \\le 1$ (Exercise 7's corollary).\n", "3. So $P(A \\cap B) \\ge P(A) + P(B) - 1$. Combined with $P(A \\cap B) \\ge 0$ this gives the lower Fréchet bound $\\max\\{0, P(A)+P(B)-1\\}$. The upper bound $\\min\\{P(A), P(B)\\}$ follows from monotonicity, since $A \\cap B \\subseteq A$ and $A \\cap B \\subseteq B$.\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Randomised assignment\n", "\n", "A diet study recruits 1,000 people and assigns 450 to treatment. What is the probability that a randomly chosen participant is in the control group?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Share assigned to treatment:\n", "\n", "$$\n", "P(T) = \\frac{450}{1000} = 0.45.\n", "$$\n", "\n", "**Step 2.** Every participant is in exactly one arm, so control is the complement of treatment:\n", "\n", "$$\n", "P(T^c) = 1 - P(T) = 1 - 0.45 = 0.55.\n", "$$\n", "\n", "**Answer:** 0.55.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Training and promotion\n", "\n", "In a labour dataset, $P(\\text{trained}) = 0.30$, $P(\\text{promoted}) = 0.20$ and $P(\\text{trained and promoted}) = 0.10$. Find the probability that a worker was trained or promoted, and the probability of neither.\n", "\n", "
Show solution\n", "\n", "Let $A$ = trained and $B$ = promoted.\n", "\n", "**Step 1.** Inclusion–exclusion (Exercise 9):\n", "\n", "$$\n", "P(A \\cup B) = P(A) + P(B) - P(A \\cap B) = 0.30 + 0.20 - 0.10 = 0.40.\n", "$$\n", "\n", "**Step 2.** \"Neither\" means not trained and not promoted. By De Morgan, $A^c \\cap B^c = (A \\cup B)^c$, so\n", "\n", "$$\n", "P(A^c \\cap B^c) = 1 - P(A \\cup B) = 1 - 0.40 = 0.60.\n", "$$\n", "\n", "**Answer:** 40% were trained or promoted; 60% were neither.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · What the marginals can tell you\n", "\n", "Administrative data show that 70% of households are *eligible* for a subsidy and 50% *applied*. The two files cannot be linked. What can you say about the share that are both eligible and applied?\n", "\n", "
Show solution\n", "\n", "Let $A$ = eligible ($P(A) = 0.70$) and $B$ = applied ($P(B) = 0.50$).\n", "\n", "**Step 1.** Lower Fréchet bound (Exercise 10):\n", "\n", "$$\n", "P(A \\cap B) \\ge \\max\\{0,\\; 0.70 + 0.50 - 1\\} = 0.20.\n", "$$\n", "\n", "**Step 2.** Upper Fréchet bound (monotonicity, since $A \\cap B$ sits inside both events):\n", "\n", "$$\n", "P(A \\cap B) \\le \\min\\{0.70,\\; 0.50\\} = 0.50.\n", "$$\n", "\n", "**Step 3.** For comparison, if eligibility and application were independent: $0.70 \\times 0.50 = 0.35$.\n", "\n", "**Answer:** between 20% and 50%. Without linked records the share is only *partially identified*; the 35% figure needs an independence assumption the data cannot check.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Two interventions that might fail\n", "\n", "A city runs two traffic interventions. Intervention 1 fails with probability 0.05 and intervention 2 with probability 0.08.\n", "\n", "(a) Without knowing how the failures are related, what is the largest and smallest possible probability that at least one fails?\n", "(b) What is it if failures are independent?\n", "\n", "
Show solution\n", "\n", "Let $F_1$ and $F_2$ be the failure events.\n", "\n", "**(a) No information about the overlap**\n", "\n", "**Step 1.** Largest value, from Boole's inequality (reached when the two failures never happen together):\n", "\n", "$$\n", "P(F_1 \\cup F_2) \\le P(F_1) + P(F_2) = 0.05 + 0.08 = 0.13.\n", "$$\n", "\n", "**Step 2.** Smallest value, from monotonicity (reached when $F_1 \\subseteq F_2$, i.e. whenever intervention 1 fails, 2 fails too):\n", "\n", "$$\n", "P(F_1 \\cup F_2) \\ge \\max\\{0.05,\\; 0.08\\} = 0.08.\n", "$$\n", "\n", "**(b) Independent failures**\n", "\n", "**Step 3.** Under independence, $P(F_1 \\cap F_2) = 0.05 \\times 0.08 = 0.004$.\n", "\n", "**Step 4.** Inclusion–exclusion:\n", "\n", "$$\n", "P(F_1 \\cup F_2) = 0.05 + 0.08 - 0.004 = 0.126.\n", "$$\n", "\n", "Check via the complement: $1 - P(\\text{both succeed}) = 1 - (0.95)(0.92) = 0.126$.\n", "\n", "**Answer:** (a) between 0.08 and 0.13; (b) 0.126.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Counting assignments\n", "\n", "An experiment has 6 villages; exactly 3 are treated, chosen completely at random.\n", "\n", "(a) How many assignments are possible?\n", "(b) What is the probability that village 1 is treated?\n", "(c) What is the probability that villages 1 and 2 are both treated?\n", "\n", "
Show solution\n", "\n", "**(a) Number of assignments**\n", "\n", "**Step 1.** Choose which 3 of the 6 villages are treated:\n", "\n", "$$\n", "\\binom{6}{3} = \\frac{6!}{3!\\,3!} = 20.\n", "$$\n", "\n", "**(b) Village 1 treated**\n", "\n", "**Step 2.** Fix village 1 as treated; the other 2 treated villages come from the remaining 5: $\\binom{5}{2} = 10$ assignments.\n", "\n", "**Step 3.** All 20 assignments are equally likely, so $P = 10/20 = 0.5$.\n", "\n", "**(c) Villages 1 and 2 both treated**\n", "\n", "**Step 4.** Fix both as treated; the last treated village comes from the remaining 4: $\\binom{4}{1} = 4$ assignments.\n", "\n", "**Step 5.** $P = 4/20 = 0.2$.\n", "\n", "**Answer:** 20 assignments; 0.5; 0.2. Note that 0.2 is *not* $0.5 \\times 0.5 = 0.25$: under complete randomisation, treating one village leaves fewer treated slots for the others, so assignments are slightly negatively dependent.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import itertools\n", "import numpy as np\n", "from math import comb\n", "\n", "# Exercise 15: enumerate every complete-randomisation assignment\n", "assignments = list(itertools.combinations(range(6), 3))\n", "print(\"number of assignments:\", len(assignments), \"=\", comb(6, 3))\n", "p1 = np.mean([0 in a for a in assignments])\n", "p12 = np.mean([(0 in a) and (1 in a) for a in assignments])\n", "print(f\"P(village 1 treated) = {p1:.2f}; P(villages 1 and 2 treated) = {p12:.2f}\")\n", "\n", "# Exercises 12-14: inclusion-exclusion and bounds checked by simulation\n", "rng = np.random.default_rng(1)\n", "n = 1_000_000\n", "f1 = rng.random(n) < 0.05\n", "f2 = rng.random(n) < 0.08 # independent of f1\n", "print(f\"P(at least one fails), independent: {np.mean(f1 | f2):.4f} (theory 0.126)\")\n", "\n", "# Frechet bounds for Exercise 13\n", "pa, pb = 0.70, 0.50\n", "print(\"Frechet bounds:\", round(max(0, pa + pb - 1), 2), \"to\", min(pa, pb))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- Three axioms generate every probability rule you will use in this course.\n", "- Inclusion–exclusion links unions and intersections; Boole and Fréchet bound what you cannot observe.\n", "- Complete randomisation defines a finite, equally likely probability space over assignments." ] } ] }