Day 35: Randomisation & Identification · Math for Causal Inference

Theory

Assignment mechanisms

Design	Rule	Number treated
Bernoulli	each unit independently with probability $p$	random
Complete	exactly $n_1$ of $n$ , all $\binom{n}{n_1}$ sets equally likely	fixed
Stratified (blocked)	complete randomisation within blocks	fixed per block (Day 38)
Cluster	whole groups assigned together	Day 40

All share three properties (Imbens & Rubin 2015): individualistic (probabilities do not depend on other units' outcomes), probabilistic ( $0 < P(T_i = 1) < 1$ ) and unconfounded (assignment does not depend on potential outcomes).

Why randomisation identifies the ATE

Random assignment makes $T$ independent of $(Y(0), Y(1), X)$ , all of which are fixed before assignment. Hence

E[Y \mid T = 1] - E[Y \mid T = 0] = E[Y(1)] - E[Y(0)] = \text{ATE}.

Selection bias and heterogeneity bias (Day 34) are zero by design. This is identification without any model of outcomes.

Balance

Independence implies balance in expectation: the treated and control groups have the same covariate distribution on average across randomisations. In any one experiment, chance imbalances occur. A balance table is a check that randomisation was implemented, not a test of whether it "worked": with 20 covariates, about one will differ "significantly" at 5% by chance alone. Large chance imbalance in a strong predictor of the outcome is best handled by design (blocking, Day 38) or pre-specified adjustment (Day 39).

Internal and external validity

Internal validity: the estimate is causal for the units studied. Random assignment delivers it.
External validity: the estimate applies to other populations, places or scales. Random sampling from the target population helps; random assignment does not.

Part 1 · Warm-up

Exercise 1 · Bernoulli surprises

Ten clinics are each assigned to a new protocol by a fair coin flip. What is the probability that none is treated?

Show solution

Step 1. Each clinic is untreated with probability 0.5, independently.

Step 2. $0.5^{10} = 1/1024 \approx 0.001$ .

Answer: about 0.1%; small, but Bernoulli designs can produce lopsided splits, which complete randomisation rules out.

Exercise 2 · Counting complete randomisations

A study randomises 10 of 20 schools to a reading programme. How many assignments are possible?

Show solution

Step 1. $\binom{20}{10} = \frac{20!}{10!\,10!} = 184{,}756$ .

Answer: 184,756 equally likely assignments.

Exercise 3 · How many get treated?

200 farmers are each offered a subsidy with probability 0.3 (Bernoulli). Find the mean and sd of the number offered.

Show solution

Step 1. Number offered $\sim$ Binomial(200, 0.3): mean $200 \times 0.3 = 60$ .

Step 2. sd $= \sqrt{200 \times 0.3 \times 0.7} = \sqrt{42} \approx 6.5$ .

Answer: about 60 ± 6.5.

Exercise 4 · Balance tests by chance

A balance table compares treatment and control on 20 independent baseline covariates at the 5% level. How many "significant" differences should you expect, and what is the chance of at least one?

Show solution

Step 1. Expected number: $20 \times 0.05 = 1$ .

Step 2. At least one: $1 - 0.95^{20} \approx 0.64$ .

Answer: one, and a 64% chance of at least one, even with perfect randomisation.

Exercise 5 · Lotteries

300 families apply for 100 places in an oversubscribed charter school, allocated by lottery. What is each applicant's probability of admission, and who is the study population?

Show solution

Step 1. $P(\text{win}) = 100/300 = 1/3$ for every applicant.

Step 2. The comparison is winners versus losers among applicants.

Answer: 1/3. The lottery identifies the effect for families who apply, who may differ from non-applicants (external validity).

Part 2 · Proofs

Exercise 6 · Randomisation identifies the ATE

Prove that if $T$ is independent of $(Y(0), Y(1))$ and $0 < P(T = 1) < 1$ , then $E[Y \mid T = 1] - E[Y \mid T = 0] = \text{ATE}$ .

Show solution

Step 1. $E[Y \mid T = 1] = E[Y(1) \mid T = 1]$ (observed outcome of the treated).

Step 2. Independence: $E[Y(1) \mid T = 1] = E[Y(1)]$ .

Step 3. Likewise $E[Y \mid T = 0] = E[Y(0) \mid T = 0] = E[Y(0)]$ . ( $0 < P(T = 1) < 1$ makes both conditional means defined.)

Step 4. Difference: $E[Y(1)] - E[Y(0)] = \text{ATE}$ . $\blacksquare$

Exercise 7 · Dependence in complete randomisation

With $n_1$ of $n$ treated completely at random, find $P(T_i = 1)$ , $P(T_i = 1, T_j = 1)$ for $i \ne j$ , and $\operatorname{Cov}(T_i, T_j)$ .

Show solution

Step 1. By symmetry each unit is equally likely to be among the $n_1$ : $P(T_i = 1) = n_1/n$ .

Step 2. Given $i$ is treated, $n_1 - 1$ of the remaining $n - 1$ slots are treated: $P(T_i = 1, T_j = 1) = \frac{n_1}{n}\cdot\frac{n_1 - 1}{n - 1}$ .

Step 3. $\operatorname{Cov} = \frac{n_1(n_1 - 1)}{n(n - 1)} - \frac{n_1^2}{n^2} = \frac{n_1}{n}\Big(\frac{n_1 - 1}{n - 1} - \frac{n_1}{n}\Big) = -\frac{n_1n_0}{n^2(n - 1)}$ .

Answer: slightly negative covariance (Day 1, Exercise 15); it drives the finite-population variance of Day 36.

Exercise 8 · Balance in expectation

Under complete randomisation, prove $E[\bar X_1 - \bar X_0] = 0$ for any fixed covariate $X$ .

Show solution

Step 1. $\bar X_1 = \frac{1}{n_1}\sum_iT_iX_i$ , so $E[\bar X_1] = \frac{1}{n_1}\sum_i\frac{n_1}{n}X_i = \bar X$ .

Step 2. Likewise $E[\bar X_0] = \bar X$ .

Step 3. Difference: 0. $\blacksquare$

Covariates behave like potential outcomes that treatment cannot change (Day 31, Exercise 8 with $Y(1) = Y(0) = X$ ).

Exercise 9 · Unbiased on average, not in every draw

Suppose $Y_i(0) = X_i + \varepsilon_i$ and the effect is constant, $\tau$ . Show that, conditional on the realised assignment, $\bar Y_1 - \bar Y_0 = \tau + (\bar X_1 - \bar X_0) + (\bar\varepsilon_1 - \bar\varepsilon_0)$ . What does this imply when you observe a large baseline imbalance?

Show solution

Step 1. Treated outcomes: $Y_i = X_i + \varepsilon_i + \tau$ ; control outcomes: $Y_i = X_i + \varepsilon_i$ .

Step 2. Average within each group and subtract: $\tau + (\bar X_1 - \bar X_0) + (\bar\varepsilon_1 - \bar\varepsilon_0)$ .

Step 3. Across randomisations the imbalance terms average to zero (Exercise 8), so the estimator is unbiased.

Step 4. But in this experiment, if $\bar X_1 - \bar X_0$ is large, the estimate is shifted by it.

Answer: a large observed imbalance in a predictive covariate tells you this particular estimate is likely off; adjusting for $X$ (Day 39) or blocking on it in advance (Day 38) fixes this.

Exercise 10 · Unequal probabilities

Pupils in poor districts are treated with probability 0.8 and others with 0.2. Explain why a simple difference in means is biased for the ATE and how weighting by $1/P(T_i = t)$ fixes it.

Show solution

Step 1. The treated group over-represents poor districts (80% vs 20% treated), so if $Y(0)$ differs by district, the comparison mixes in district differences: selection bias by design.

Step 2. Within each district, assignment is random, so district-specific differences are unbiased.

Step 3. Weighting each unit by the inverse of its probability of the treatment it got re-creates the population in both arms: $E[TY/e] = E[Y(1)]$ and $E[(1 - T)Y/(1 - e)] = E[Y(0)]$ (Day 14).

Answer: use IPW (or estimate by district and average with population shares). Known probabilities make this exact; Day 44 handles estimated ones.

Part 3 · Applied

Exercise 11 · A charter-school lottery

Among applicants, lottery winners score 0.25 sd higher in maths than losers two years later (SE 0.08). All winners enrol and no losers do. Interpret the estimate and its CI.

Show solution

Step 1. The lottery makes winning independent of potential scores, so 0.25 is an unbiased estimate of the ATE of charter attendance for applicants.

Step 2. 95% CI: $0.25 \pm 1.96 \times 0.08 = [0.09,\; 0.41]$ .

Answer: attending raised maths scores by about a quarter of a standard deviation (plausibly 0.1–0.4) for families who applied; whether it would do so for non-applicants is an external-validity question.

Exercise 12 · A vaccine trial

30,000 adults are randomised 2:1 to vaccine or placebo. There are 11 infections among 20,000 vaccinated and 185 among 10,000 on placebo. Compute the infection risks, the risk difference and vaccine efficacy.

Show solution

Step 1. Risks: vaccine $11/20{,}000 = 0.00055$ ; placebo $185/10{,}000 = 0.0185$ .

Step 2. Risk difference $= 0.00055 - 0.0185 \approx -0.018$ (18 fewer infections per 1,000).

Step 3. Efficacy $= 1 - 0.00055/0.0185 \approx 1 - 0.030 = 0.97$ .

Answer: 97% efficacy. Randomisation means the arms differ only by vaccine (and chance), so the comparison is causal.

Exercise 13 · Microcredit in one city

An RCT finds that offering microloans in one large city raised business investment by 20%. A ministry plans a national roll-out to rural areas. What does randomisation guarantee, and what does it not?

Show solution

Step 1. Internal validity: within the study city, the 20% is causal for the households studied.

Step 2. External validity is not guaranteed: rural households differ in business opportunities, market access and existing credit.

Step 3. Scale effects matter too: a national programme can change interest rates or local demand (SUTVA, Day 32).

Answer: randomisation guarantees the first; the roll-out needs evidence from more sites, a model of heterogeneity (CATEs, Day 33) or a pilot in rural areas.

Exercise 14 · An unlucky balance table

In a job-training RCT, treated workers earned $500 more at baseline than controls (SE $400). The follow-up difference in earnings is $1,200. What should the analyst do?

Show solution

Step 1. The baseline difference has $z = 500/400 = 1.25$ : consistent with chance, so no evidence that randomisation failed.

Step 2. But baseline earnings strongly predict follow-up earnings, so part of the $1,200 may reflect the imbalance (Exercise 9).

Step 3. Report the pre-specified covariate-adjusted estimate (Day 39) alongside the raw difference; do not drop or re-weight units ad hoc.

Answer: keep the experiment, adjust for baseline earnings, and report both estimates.

Exercise 15 · Randomisation as the basis for inference

Fisher argued that randomisation provides "the reasoned basis" for inference. In a study of 8 villages with 4 treated, how many assignments could have happened, and what probability statement does this enable without any model of outcomes?

Show solution

Step 1. $\binom84 = 70$ equally likely assignments.

Step 2. Under the sharp null of no effect for any village, every outcome is known whatever the assignment, so we can compute the test statistic for all 70.

Step 3. The p-value is the share of the 70 assignments giving a statistic at least as extreme as the one observed.

Answer: 70 assignments; randomisation alone yields an exact p-value (Day 37).

Verify in Python

import numpy as np
from math import comb

print("assignments C(20,10):", comb(20, 10))
print("P(none treated) with 10 coin flips:", 0.5**10)

# Balance in expectation and chance imbalance across randomisations
rng = np.random.default_rng(35)
n, n1 = 100, 50
X = rng.lognormal(10, 0.5, n)                       # baseline earnings, fixed
y0 = X / 1000 + rng.normal(0, 3, n); y1 = y0 + 2    # constant effect of 2
diffs, ests = [], []
for _ in range(20_000):
    T = np.zeros(n, bool); T[rng.choice(n, n1, replace=False)] = True
    diffs.append(X[T].mean() - X[~T].mean())
    ests.append(y1[T].mean() - y0[~T].mean())
diffs, ests = np.array(diffs), np.array(ests)
print(f"baseline imbalance: mean {diffs.mean():.0f}, sd {diffs.std():.0f} (mean ~0 relative to sd); mean estimate {ests.mean():.3f} (truth 2)")
print("corr(imbalance, estimate):", round(np.corrcoef(diffs, ests)[0, 1], 2))

# Vaccine trial (Exercise 12)
r1, r0 = 11 / 20_000, 185 / 10_000
print(f"risks {r1:.5f} vs {r0:.4f}; efficacy {1 - r1 / r0:.3f}")

Key takeaways

Random assignment makes treatment independent of potential outcomes and covariates, so the difference in means identifies the ATE.
Balance holds in expectation; chance imbalance in any one experiment is handled by blocking or pre-specified adjustment.
Randomisation buys internal validity; external validity needs more.

Day 35. Randomisation & Identification

Theory

Assignment mechanisms

Why randomisation identifies the ATE

Balance

Internal and external validity

Part 1 · Warm-up

Exercise 1 · Bernoulli surprises

Exercise 2 · Counting complete randomisations

Exercise 3 · How many get treated?

Exercise 4 · Balance tests by chance

Exercise 5 · Lotteries

Part 2 · Proofs

Exercise 6 · Randomisation identifies the ATE

Exercise 7 · Dependence in complete randomisation

Exercise 8 · Balance in expectation

Exercise 9 · Unbiased on average, not in every draw

Exercise 10 · Unequal probabilities

Part 3 · Applied

Exercise 11 · A charter-school lottery

Exercise 12 · A vaccine trial

Exercise 13 · Microcredit in one city

Exercise 14 · An unlucky balance table

Exercise 15 · Randomisation as the basis for inference

Verify in Python

Key takeaways

Simulation Lab: Randomisation Balances Covariates