Day 4: Common Distributions & the Data-Generating Process

Theory

A small toolkit of distributions

Distribution	Typical use	Support	Mean	Variance
Bernoulli( $p$ )	treated or not, employed or not	$\{0, 1\}$	$p$	$p(1-p)$
Binomial( $n, p$ )	successes in $n$ trials	$\{0, \dots, n\}$	$np$	$np(1-p)$
Poisson( $\lambda$ )	counts: patents, visits, claims	$\{0, 1, \dots\}$	$\lambda$	$\lambda$
Uniform( $a, b$ )	random draws, lotteries	$[a, b]$	$\tfrac{a+b}{2}$	$\tfrac{(b-a)^2}{12}$
Normal( $\mu, \sigma^2$ )	test scores, errors, estimators	$\mathbb{R}$	$\mu$	$\sigma^2$
Exponential( $\lambda$ )	durations: unemployment spells	$[0, \infty)$	$1/\lambda$	$1/\lambda^2$
Log-normal( $\mu, \sigma^2$ )	wages, firm size, health costs	$(0, \infty)$	$e^{\mu + \sigma^2/2}$	$(e^{\sigma^2} - 1)e^{2\mu + \sigma^2}$

Key formulas:

P(X = k) = \binom{n}{k}p^k(1-p)^{n-k}, \qquad P(X = k) = \frac{e^{-\lambda}\lambda^k}{k!}, \qquad f(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}.

Any normal variable can be standardised: $Z = (X - \mu)/\sigma \sim N(0, 1)$ , and $\Phi$ denotes the standard normal CDF. Independent normals add: if $X \sim N(\mu_1, \sigma_1^2)$ and $Y \sim N(\mu_2, \sigma_2^2)$ are independent, $X + Y \sim N(\mu_1 + \mu_2, \sigma_1^2 + \sigma_2^2)$ .

The data-generating process

A DGP is a recipe that says how each variable is produced. Writing one forces you to state your assumptions, and simulating it lets you check whether an estimator recovers the truth. A causal DGP specifies potential outcomes and how treatment is assigned:

\begin{aligned} U &\sim \text{Bernoulli}(0.5) && \text{(unobserved ability)}\\ T \mid U &\sim \text{Bernoulli}(0.2 + 0.6U) && \text{(who takes the programme)}\\ Y(t) &= 10 + 5t + 20U + \varepsilon, \quad \varepsilon \sim N(0, 1) && \text{(potential outcomes)}\\ Y &= Y(T) && \text{(what we observe)} \end{aligned}

Here the true effect is 5 for everyone. Whether a naive comparison recovers 5 depends on the second line (Exercise 15).

Causal link. Every method in this course will be checked against a simulated DGP whose true effect you know. Learning to write DGPs today is learning how to test causal estimators.

Part 1 · Warm-up

Exercise 1 · Binomial attendance

Each of 10 training sessions is attended with probability 0.3, independently. Find $P(X = 3)$ , $E[X]$ and $\operatorname{Var}(X)$ .

Show solution

Step 1. $X \sim \text{Binomial}(n = 10, p = 0.3)$ . The number of ways to pick which 3 sessions are attended: $\binom{10}{3} = 120$ .

Step 2. Probability of one particular pattern with 3 attended and 7 missed: $0.3^3 \times 0.7^7 = 0.027 \times 0.08235 \approx 0.002224$ .

Step 3. Multiply: $P(X = 3) = 120 \times 0.002224 \approx 0.2668$ .

Step 4. Moments: $E[X] = np = 3$ and $\operatorname{Var}(X) = np(1 - p) = 10(0.3)(0.7) = 2.1$ .

Answer: $P(X = 3) \approx 0.267$ , mean 3 sessions, variance 2.1.

Exercise 2 · Poisson claims

Insurance claims per month are Poisson with $\lambda = 4$ . Find $P(X = 0)$ and $P(X \le 1)$ .

Show solution

Step 1. Poisson pmf: $P(X = k) = e^{-\lambda}\lambda^k/k!$ with $\lambda = 4$ .

Step 2. $P(X = 0) = e^{-4} \approx 0.0183$ .

Step 3. $P(X = 1) = e^{-4}\cdot 4 \approx 0.0733$ .

Step 4. $P(X \le 1) = 0.0183 + 0.0733 = 5e^{-4} \approx 0.0916$ .

Answer: a claim-free month has probability 1.8%; at most one claim, 9.2%.

Exercise 3 · Standardising wages

Wages are $N(50{,}000,\; 10{,}000^2)$ . What share earn more than $65,000?

Show solution

Step 1. Standardise: $z = (65{,}000 - 50{,}000)/10{,}000 = 1.5$ .

Step 2. $P(W > 65{,}000) = P(Z > 1.5) = 1 - \Phi(1.5) = 1 - 0.9332 = 0.0668$ .

Answer: about 6.7% of workers earn more than $65,000.

Exercise 4 · Unemployment duration

Spell length (months) is exponential with mean 2, i.e. $\lambda = 0.5$ . What is $P(T > 3)$ ?

Show solution

For the exponential, $P(T > t) = e^{-\lambda t}$ , so $P(T > 3) = e^{-1.5} \approx 0.223$ .

Exercise 5 · A uniform lottery

A lottery assigns a voucher uniformly between $20 and $40. Find the mean and variance.

Show solution

Mean $(20 + 40)/2 = 30$ ; variance $(40 - 20)^2/12 = 400/12 \approx 33.3$ .

Part 2 · Proofs

Exercise 6 · Binomial moments from Bernoullis

Write $X = \sum_{i=1}^n B_i$ with independent $B_i \sim \text{Bernoulli}(p)$ . Derive $E[X]$ and $\operatorname{Var}(X)$ .

Show solution

By linearity, $E[X] = \sum_i E[B_i] = np$ . Because the $B_i$ are independent, their covariances are zero (Day 5), so variances add: $\operatorname{Var}(X) = \sum_i p(1-p) = np(1-p)$ .

Exercise 7 · The Poisson mean

Show that $E[X] = \lambda$ for $X \sim \text{Poisson}(\lambda)$ .

Show solution

E[X] = \sum_{k=0}^\infty k\frac{e^{-\lambda}\lambda^k}{k!} = \lambda e^{-\lambda}\sum_{k=1}^\infty \frac{\lambda^{k-1}}{(k-1)!} = \lambda e^{-\lambda}\sum_{j=0}^\infty \frac{\lambda^{j}}{j!} = \lambda e^{-\lambda}e^{\lambda} = \lambda.

Exercise 8 · Memorylessness

Prove that the exponential distribution satisfies $P(T > s + t \mid T > s) = P(T > t)$ .

Show solution

P(T > s + t \mid T > s) = \frac{P(T > s + t)}{P(T > s)} = \frac{e^{-\lambda(s+t)}}{e^{-\lambda s}} = e^{-\lambda t} = P(T > t).

A worker unemployed for 6 months is no more likely to find a job next month than one just laid off. Real data usually show duration dependence, which is evidence against the exponential model.

Exercise 9 · Sums and differences of normals

$X \sim N(1, 4)$ and $Y \sim N(2, 9)$ are independent. Find the distributions of $X + Y$ and $X - Y$ .

Show solution

Step 1. $X + Y \sim N(3, 13)$ .

Step 2. For the difference, $\operatorname{Var}(X - Y) = \operatorname{Var}(X) + (-1)^2\operatorname{Var}(Y) = 13$ , so $X - Y \sim N(-1, 13)$ .

Step 3. Variances add even when you subtract, which is why the standard error of a difference in means is larger than either group's.

Exercise 10 · The log-normal mean

If $\ln Y \sim N(\mu, \sigma^2)$ , show that $E[Y] = e^{\mu + \sigma^2/2}$ .

Show solution

Step 1. Let $X = \ln Y$ .

Step 2. Then $E[Y] = E[e^X] = \int e^x \frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/(2\sigma^2)}dx$ .

Step 3. Complete the square in the exponent:

x - \frac{(x-\mu)^2}{2\sigma^2} = -\frac{\big(x - (\mu + \sigma^2)\big)^2}{2\sigma^2} + \mu + \frac{\sigma^2}{2}.

Step 4. The remaining integral is a $N(\mu + \sigma^2, \sigma^2)$ density, which integrates to 1, leaving $E[Y] = e^{\mu + \sigma^2/2}$ .

Step 5. Note $E[Y] > e^{E[\ln Y]} = e^{\mu}$ , as Jensen requires.

Part 3 · Applied

Exercise 11 · Rare events

1,000 firms each go bankrupt this year with probability 0.002, independently. Compare the exact binomial probability of no bankruptcies with the Poisson approximation.

Show solution

Step 1 (exact). $X \sim \text{Binomial}(1000, 0.002)$ , so $P(X = 0) = 0.998^{1000}$ .

Step 2. Evaluate with logs: $1000\ln 0.998 = 1000 \times (-0.002002) = -2.002$ , so $P(X = 0) = e^{-2.002} \approx 0.1350$ .

Step 3 (approximation). $\lambda = np = 1000 \times 0.002 = 2$ , so the Poisson gives $P(X = 0) = e^{-2} \approx 0.1353$ .

Answer: 0.1350 versus 0.1353. With many firms and a small probability each, the Poisson is an excellent approximation.

Exercise 12 · Overdispersion

Doctor visits have sample mean 3 and sample variance 12. Is a Poisson model reasonable?

Show solution

Step 1. A Poisson model forces $\operatorname{Var}(X) = E[X]$ .

Step 2. Here the variance is $12/3 = 4$ times the mean: overdispersion.

Step 3. Likely cause: unobserved differences between people (health, insurance, distance to the clinic) make the visit rate vary across individuals, which adds variance.

Answer: a pure Poisson model is not reasonable. Use a negative binomial model, or Poisson regression with robust standard errors.

Exercise 13 · Mean versus median wage

Log wages are $N(10, 0.5^2)$ . Find the median and mean wage.

Show solution

Step 1. $\ln Y$ is normal, so its median equals its mean, 10. Because $\exp$ is increasing, it maps the median of $\ln Y$ to the median of $Y$ :

\text{median}(Y) = e^{10} \approx 22{,}026.

Step 2. Mean of a log-normal (Exercise 10) with $\mu = 10$ , $\sigma^2 = 0.25$ :

E[Y] = e^{10 + 0.25/2} = e^{10.125} \approx 24{,}959.

Answer: median ≈ $22,026 and mean ≈ $24,959. Right-skewed variables like wages have a mean above the median.

Exercise 14 · A randomised tutoring programme

A school district simulates a tutoring RCT. A pupil's test score without tutoring is $Y(0) \sim N(50, 10^2)$ ; tutoring adds 5 points to everyone, $Y(1) = Y(0) + 5$ ; and a coin flip assigns tutoring, $T \sim \text{Bernoulli}(0.5)$ , independently of $Y(0)$ . Find $E[Y \mid T = 1] - E[Y \mid T = 0]$ and $\operatorname{Var}(Y)$ for the observed score $Y$ .

Show solution

Step 1. Observed score: $Y = Y(0) + 5T$ .

Step 2. Because $T$ is independent of $Y(0)$ , $E[Y(0) \mid T = t] = 50$ for both arms. So

E[Y \mid T = 1] = 50 + 5 = 55, \qquad E[Y \mid T = 0] = 50.

Step 3. The difference is $55 - 50 = 5$ , the true effect.

Step 4. Variance of a sum of independent pieces: $\operatorname{Var}(Y) = \operatorname{Var}(Y(0)) + 5^2\operatorname{Var}(T) = 100 + 25(0.25) = 106.25$ .

Answer: the difference in means is 5 (unbiased for the effect), and $\operatorname{Var}(Y) = 106.25$ .

Exercise 15 · A confounded job-training DGP

Use the DGP in the theory block, read as job training: $U$ is high motivation ( $U \sim \text{Bern}(0.5)$ ), motivated workers enrol more often, $P(T = 1 \mid U) = 0.2 + 0.6U$ , and earnings (in $1,000) are $Y = 10 + 5T + 20U + \varepsilon$ . Compute $E[Y \mid T = 1] - E[Y \mid T = 0]$ and the selection bias.

Show solution

Step 1. Share enrolled: $P(T = 1) = 0.5(0.8) + 0.5(0.2) = 0.5$ .

Step 2. Bayes' theorem for motivation among trainees and non-trainees:

P(U = 1 \mid T = 1) = \frac{0.8 \times 0.5}{0.5} = 0.8, \qquad P(U = 1 \mid T = 0) = \frac{0.2 \times 0.5}{0.5} = 0.2.

Step 3. Conditional means ( $E[\varepsilon] = 0$ ):

E[Y \mid T = 1] = 10 + 5 + 20(0.8) = 31, \qquad E[Y \mid T = 0] = 10 + 20(0.2) = 14.

Step 4. Naive difference: $31 - 14 = 17$ . True effect: 5. Selection bias: $17 - 5 = 12$ , which is $20 \times (0.8 - 0.2)$ .

Answer: the naive comparison says training is worth $17k; $12k of that is motivated workers selecting into training.

Verify in Python

import numpy as np
from scipy import stats

print("Binomial P(X=3):", round(stats.binom.pmf(3, 10, 0.3), 4))
print("Poisson P(X<=1):", round(stats.poisson.cdf(1, 4), 4))
print("Normal tail:", round(stats.norm.sf(65_000, 50_000, 10_000), 4))
print("Exact vs Poisson:", round(0.998**1000, 4), round(np.exp(-2), 4))
print("Log-normal mean:", round(np.exp(10 + 0.5**2 / 2)))

# Exercise 15: simulate the confounded DGP
rng = np.random.default_rng(42)
n = 1_000_000
U = rng.binomial(1, 0.5, n)
T = rng.binomial(1, 0.2 + 0.6 * U)
Y = 10 + 5 * T + 20 * U + rng.normal(0, 1, n)
print("naive difference:", round(Y[T == 1].mean() - Y[T == 0].mean(), 2), "(true effect 5)")

Key takeaways

A few distributions cover most data: Bernoulli/binomial for binary, Poisson for counts, exponential for durations, normal and log-normal for continuous outcomes.
A DGP makes assumptions explicit and gives you a ground truth to test estimators against.
When an unobserved variable drives both treatment and outcome, a naive comparison mixes the effect with selection bias.

Day 4. Common Distributions & the Data-Generating Process

Theory

A small toolkit of distributions

The data-generating process

Part 1 · Warm-up

Exercise 1 · Binomial attendance

Exercise 2 · Poisson claims

Exercise 3 · Standardising wages

Exercise 4 · Unemployment duration

Exercise 5 · A uniform lottery

Part 2 · Proofs

Exercise 6 · Binomial moments from Bernoullis

Exercise 7 · The Poisson mean

Exercise 8 · Memorylessness

Exercise 9 · Sums and differences of normals

Exercise 10 · The log-normal mean

Part 3 · Applied

Exercise 11 · Rare events

Exercise 12 · Overdispersion

Exercise 13 · Mean versus median wage

Exercise 14 · A randomised tutoring programme

Exercise 15 · A confounded job-training DGP

Verify in Python

Key takeaways

Simulation Lab: Distribution Explorer & DGP Sampler