Day 1: Probability Axioms & Set Theory · Math for Causal Inference

Theory

Why start here?

The fundamental problem of causal inference is that we never see the same unit both treated and untreated. Every causal estimate is therefore a statement about populations, and populations are described with probability. Before we can define an average treatment effect we need to be exact about what "probability" means.

Sample space, events and set operations

The sample space $\Omega$ is the set of all possible outcomes of a random experiment.
An event $A$ is a subset of $\Omega$ . The collection of events we are allowed to measure is written $\mathcal{F}$ .
For events $A$ $A$ and $B$ $B$ :
- union $A \cup B$ : " $A$ or $B$ (or both)"
- intersection $A \cap B$ : "both $A$ and $B$ "
- complement $A^c = \Omega \setminus A$ : "not $A$ "
- $A$ and $B$ are mutually exclusive (disjoint) if $A \cap B = \emptyset$ .
De Morgan's laws: $(A \cup B)^c = A^c \cap B^c$ and $(A \cap B)^c = A^c \cup B^c$ .

Kolmogorov's axioms

A probability measure $P$ assigns a number to each event so that:

Non-negativity: $P(A) \ge 0$ for every event $A$ .
Normalisation: $P(\Omega) = 1$ .
Countable additivity: if $A_1, A_2, \dots$ are pairwise disjoint, then

P\left(\bigcup_{i=1}^{\infty} A_i\right) = \sum_{i=1}^{\infty} P(A_i).

Everything else in probability is derived from these three lines. Today you will derive the most useful consequences yourself:

Rule	Statement
Empty set	$P(\emptyset) = 0$
Complement	$P(A^c) = 1 - P(A)$
Monotonicity	$A \subseteq B \Rightarrow P(A) \le P(B)$
Inclusion–exclusion	$P(A \cup B) = P(A) + P(B) - P(A \cap B)$
Boole's inequality	$P(A \cup B) \le P(A) + P(B)$
Fréchet bounds	$\max\{0,\, P(A)+P(B)-1\} \le P(A \cap B) \le \min\{P(A), P(B)\}$

Equally likely outcomes and counting

When $\Omega$ is finite and every outcome is equally likely, $P(A) = |A| / |\Omega|$ . In a completely randomised experiment with $n$ units of which exactly $n_1$ are treated, every one of the $\binom{n}{n_1}$ possible assignments has the same probability. This is the probability space behind Fisher's randomisation test (Phase 3).

Causal link. The Fréchet bounds are your first taste of partial identification: when the data pin down only the marginals, the joint is known only up to an interval. Manski's bounds on treatment effects (Phase 6) use exactly this logic.

Part 1 · Warm-up

Exercise 1 · Defining the sample space

An environmental regulation gives each firm three mutually exclusive options: adopt clean technology ( $C$ ), pay a carbon tax ( $T$ ), or exit the market ( $E$ ). Write the sample space $\Omega$ .

Show solution

The sample space lists every basic outcome:

\Omega = \{C, T, E\}.

Exercise 2 · Identifying an event

Using Exercise 1, write the event $A$ = "the firm stays in the market", and its complement.

Show solution

Step 1. The firm stays in the market if it adopts clean technology or pays the tax:

A = \{C, T\}.

Step 2. The complement collects every other outcome in $\Omega$ :

A^c = \Omega \setminus A = \{E\}.

Answer: $A = \{C, T\}$ and $A^c = \{E\}$ (the firm exits).

Exercise 3 · Normalisation

If $P(C) = 0.4$ and $P(E) = 0.1$ , what is $P(T)$ ?

Show solution

The three outcomes are disjoint and exhaustive, so by Axioms 2 and 3:

P(C) + P(T) + P(E) = 1 \;\Rightarrow\; P(T) = 1 - 0.4 - 0.1 = 0.5.

Exercise 4 · A tiny randomised trial

Two patients are each assigned to treatment ( $T$ ) or control ( $C$ ) by independent fair coin flips. Write $\Omega$ , the event "exactly one patient is treated", and its probability.

Show solution

Writing patient 1's arm first: $\Omega = \{TT, TC, CT, CC\}$ , four equally likely outcomes.

"Exactly one treated" is $\{TC, CT\}$ , so its probability is $2/4 = 0.5$ .

Exercise 5 · Additivity

Trial participants have no side effects ( $S_0$ ), mild ( $S_1$ ) or severe ( $S_2$ ) side effects. If $P(S_1) = 0.20$ and $P(S_2) = 0.05$ , what is the probability of any side effect? Of none?

Show solution

$S_1$ and $S_2$ are disjoint, so by Axiom 3:

P(S_1 \cup S_2) = 0.20 + 0.05 = 0.25.

"No side effect" is the complement: $P(S_0) = 1 - 0.25 = 0.75$ .

Part 2 · Proofs

Exercise 6 · The empty set

Using only the axioms, prove $P(\emptyset) = 0$ .

Show solution

$\Omega$ and $\emptyset$ are disjoint, and $\Omega \cup \emptyset = \Omega$ .
By Axiom 3, $P(\Omega) = P(\Omega \cup \emptyset) = P(\Omega) + P(\emptyset)$ .
Subtract $P(\Omega)$ (which is finite, equal to 1) from both sides: $P(\emptyset) = 0$ .

Exercise 7 · The complement rule

Prove that $P(A^c) = 1 - P(A)$ .

Show solution

$A \cap A^c = \emptyset$ and $A \cup A^c = \Omega$ .
Axiom 3 gives $P(A) + P(A^c) = P(\Omega)$ .
Axiom 2 gives $P(\Omega) = 1$ , so $P(A^c) = 1 - P(A)$ .

A corollary: since $P(A^c) \ge 0$ (Axiom 1), $P(A) \le 1$ .

Exercise 8 · Monotonicity

Prove that if $A \subseteq B$ , then $P(A) \le P(B)$ .

Show solution

Write $B = A \cup (B \cap A^c)$ . The two pieces are disjoint.
By Axiom 3, $P(B) = P(A) + P(B \cap A^c)$ .
By Axiom 1, $P(B \cap A^c) \ge 0$ , so $P(B) \ge P(A)$ .

Exercise 9 · Inclusion–exclusion

Prove $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ .

Show solution

Split the union into disjoint pieces: $A \cup B = A \cup (B \cap A^c)$ , so $P(A \cup B) = P(A) + P(B \cap A^c)$ .
Split $B$ into disjoint pieces: $B = (B \cap A) \cup (B \cap A^c)$ , so $P(B \cap A^c) = P(B) - P(A \cap B)$ .
Substitute step 2 into step 1:

P(A \cup B) = P(A) + P(B) - P(A \cap B).

Boole's inequality follows immediately, because $P(A \cap B) \ge 0$ .

Exercise 10 · The Fréchet lower bound

Prove that $P(A \cap B) \ge P(A) + P(B) - 1$ (Bonferroni's inequality).

Show solution

Rearrange inclusion–exclusion: $P(A \cap B) = P(A) + P(B) - P(A \cup B)$ .
$P(A \cup B) \le 1$ (Exercise 7's corollary).
So $P(A \cap B) \ge P(A) + P(B) - 1$ . Combined with $P(A \cap B) \ge 0$ this gives the lower Fréchet bound $\max\{0, P(A)+P(B)-1\}$ . The upper bound $\min\{P(A), P(B)\}$ follows from monotonicity, since $A \cap B \subseteq A$ and $A \cap B \subseteq B$ .

Part 3 · Applied

Exercise 11 · Randomised assignment

A diet study recruits 1,000 people and assigns 450 to treatment. What is the probability that a randomly chosen participant is in the control group?

Show solution

Step 1. Share assigned to treatment:

P(T) = \frac{450}{1000} = 0.45.

Step 2. Every participant is in exactly one arm, so control is the complement of treatment:

P(T^c) = 1 - P(T) = 1 - 0.45 = 0.55.

Answer: 0.55.

Exercise 12 · Training and promotion

In a labour dataset, $P(\text{trained}) = 0.30$ , $P(\text{promoted}) = 0.20$ and $P(\text{trained and promoted}) = 0.10$ . Find the probability that a worker was trained or promoted, and the probability of neither.

Show solution

Let $A$ = trained and $B$ = promoted.

Step 1. Inclusion–exclusion (Exercise 9):

P(A \cup B) = P(A) + P(B) - P(A \cap B) = 0.30 + 0.20 - 0.10 = 0.40.

Step 2. "Neither" means not trained and not promoted. By De Morgan, $A^c \cap B^c = (A \cup B)^c$ , so

P(A^c \cap B^c) = 1 - P(A \cup B) = 1 - 0.40 = 0.60.

Answer: 40% were trained or promoted; 60% were neither.

Exercise 13 · What the marginals can tell you

Administrative data show that 70% of households are eligible for a subsidy and 50% applied. The two files cannot be linked. What can you say about the share that are both eligible and applied?

Show solution

Let $A$ = eligible ( $P(A) = 0.70$ ) and $B$ = applied ( $P(B) = 0.50$ ).

Step 1. Lower Fréchet bound (Exercise 10):

P(A \cap B) \ge \max\{0,\; 0.70 + 0.50 - 1\} = 0.20.

Step 2. Upper Fréchet bound (monotonicity, since $A \cap B$ sits inside both events):

P(A \cap B) \le \min\{0.70,\; 0.50\} = 0.50.

Step 3. For comparison, if eligibility and application were independent: $0.70 \times 0.50 = 0.35$ .

Answer: between 20% and 50%. Without linked records the share is only partially identified; the 35% figure needs an independence assumption the data cannot check.

Exercise 14 · Two interventions that might fail

A city runs two traffic interventions. Intervention 1 fails with probability 0.05 and intervention 2 with probability 0.08.

(a) Without knowing how the failures are related, what is the largest and smallest possible probability that at least one fails? (b) What is it if failures are independent?

Show solution

Let $F_1$ and $F_2$ be the failure events.

(a) No information about the overlap

Step 1. Largest value, from Boole's inequality (reached when the two failures never happen together):

P(F_1 \cup F_2) \le P(F_1) + P(F_2) = 0.05 + 0.08 = 0.13.

Step 2. Smallest value, from monotonicity (reached when $F_1 \subseteq F_2$ , i.e. whenever intervention 1 fails, 2 fails too):

P(F_1 \cup F_2) \ge \max\{0.05,\; 0.08\} = 0.08.

(b) Independent failures

Step 3. Under independence, $P(F_1 \cap F_2) = 0.05 \times 0.08 = 0.004$ .

Step 4. Inclusion–exclusion:

P(F_1 \cup F_2) = 0.05 + 0.08 - 0.004 = 0.126.

Check via the complement: $1 - P(\text{both succeed}) = 1 - (0.95)(0.92) = 0.126$ .

Answer: (a) between 0.08 and 0.13; (b) 0.126.

Exercise 15 · Counting assignments

An experiment has 6 villages; exactly 3 are treated, chosen completely at random.

(a) How many assignments are possible? (b) What is the probability that village 1 is treated? (c) What is the probability that villages 1 and 2 are both treated?

Show solution

(a) Number of assignments

Step 1. Choose which 3 of the 6 villages are treated:

\binom{6}{3} = \frac{6!}{3!\,3!} = 20.

(b) Village 1 treated

Step 2. Fix village 1 as treated; the other 2 treated villages come from the remaining 5: $\binom{5}{2} = 10$ assignments.

Step 3. All 20 assignments are equally likely, so $P = 10/20 = 0.5$ .

(c) Villages 1 and 2 both treated

Step 4. Fix both as treated; the last treated village comes from the remaining 4: $\binom{4}{1} = 4$ assignments.

Step 5. $P = 4/20 = 0.2$ .

Answer: 20 assignments; 0.5; 0.2. Note that 0.2 is not $0.5 \times 0.5 = 0.25$ : under complete randomisation, treating one village leaves fewer treated slots for the others, so assignments are slightly negatively dependent.

Verify in Python

import itertools
import numpy as np
from math import comb

# Exercise 15: enumerate every complete-randomisation assignment
assignments = list(itertools.combinations(range(6), 3))
print("number of assignments:", len(assignments), "=", comb(6, 3))
p1 = np.mean([0 in a for a in assignments])
p12 = np.mean([(0 in a) and (1 in a) for a in assignments])
print(f"P(village 1 treated) = {p1:.2f}; P(villages 1 and 2 treated) = {p12:.2f}")

# Exercises 12-14: inclusion-exclusion and bounds checked by simulation
rng = np.random.default_rng(1)
n = 1_000_000
f1 = rng.random(n) < 0.05
f2 = rng.random(n) < 0.08          # independent of f1
print(f"P(at least one fails), independent: {np.mean(f1 | f2):.4f}  (theory 0.126)")

# Frechet bounds for Exercise 13
pa, pb = 0.70, 0.50
print("Frechet bounds:", round(max(0, pa + pb - 1), 2), "to", min(pa, pb))

Key takeaways

Three axioms generate every probability rule you will use in this course.
Inclusion–exclusion links unions and intersections; Boole and Fréchet bound what you cannot observe.
Complete randomisation defines a finite, equally likely probability space over assignments.

Day 1. Probability Axioms & Set Theory

Theory

Why start here?

Sample space, events and set operations

Kolmogorov's axioms

Equally likely outcomes and counting

Part 1 · Warm-up

Exercise 1 · Defining the sample space

Exercise 2 · Identifying an event

Exercise 3 · Normalisation

Exercise 4 · A tiny randomised trial

Exercise 5 · Additivity

Part 2 · Proofs

Exercise 6 · The empty set

Exercise 7 · The complement rule

Exercise 8 · Monotonicity

Exercise 9 · Inclusion–exclusion

Exercise 10 · The Fréchet lower bound

Part 3 · Applied

Exercise 11 · Randomised assignment

Exercise 12 · Training and promotion

Exercise 13 · What the marginals can tell you

Exercise 14 · Two interventions that might fail

Exercise 15 · Counting assignments

Verify in Python

Key takeaways

Simulation Lab: Inclusion–Exclusion & Fréchet Bounds