Day 64. 2SLS in Matrix Form
Two-stage least squares generalises IV to many instruments and controls. Project the regressors on the instruments, regress the outcome on the projections, and get the variance right. Includes the GMM interpretation and the classic mistakes (manual second-stage standard errors, the forbidden regression).
- ●Derive the 2SLS estimator with the projection matrix and show it equals OLS on first-stage fitted values.
- ●State the order and rank conditions for identification.
- ●Compute 2SLS by hand in small examples and see it as a weighted average of single-instrument estimates.
- ●Explain why manual second-stage standard errors are wrong and why nonlinear first stages must not be plugged in.
Theory
Set-up
, where () contains the endogenous regressors and the exogenous controls (including the constant). () contains the excluded instruments and the same exogenous controls. Assumptions: (instrument exogeneity) and (rank condition).
- Order condition: : at least as many instruments as endogenous regressors. : just identified; : over-identified.
The estimator
With the projection matrix ,
Two stages. (1) Regress each column of on : (exogenous columns are reproduced exactly). (2) Regress on . Because is symmetric and idempotent, , so the second-stage OLS coefficients are exactly 2SLS.
Just identified: , the IV estimator.
Variance
Under homoskedasticity, with , residuals computed with the actual . The robust (sandwich) version is
Running the second stage by hand in software uses residuals , which gives the wrong standard errors.
GMM view
2SLS minimises : GMM (Day 30) with weight matrix , which is efficient under homoskedasticity. With heteroskedasticity, two-step GMM can be more efficient.
Many instruments: a weighted average
With one endogenous regressor and orthogonal instruments, 2SLS is a weighted average of the single-instrument IV estimates, with weights proportional to the squared first-stage strength of each instrument (Exercise 11).
Two mistakes to avoid
- Controls in one stage only. Every exogenous control must appear in both stages; otherwise the first-stage residual is correlated with the omitted control.
- The forbidden regression. With a binary treatment, plugging fitted probabilities from a probit into the second stage is not 2SLS and is inconsistent unless the probit is exactly right. Use the fitted probabilities as an instrument instead, or just use the linear first stage.
Part 1 · Warm-up
Exercise 1 · Just-identified by hand
Demeaned data (no constant): , , . Compute .
Show solution
Step 1. .
Step 2. .
Step 3. .
Exercise 2 · Properties of the projection
Show that is symmetric and idempotent.
Show solution
Step 1. , since is symmetric.
Step 2. .
Exercise 3 · First-stage fitted values
In Exercise 1, compute the first-stage coefficient and fitted values .
Show solution
Step 1. .
Step 2. .
Step 3. Second stage: , matching Exercise 1.
Exercise 4 · Order condition
A wage equation has two endogenous regressors (schooling and experience) and one instrument (quarter of birth). Is it identified? What if you add age as an instrument for experience?
Show solution
Step 1. , excluded instruments 1: order condition fails; not identified.
Step 2. With age: 2 instruments for 2 endogenous regressors: just identified (if the rank condition holds: age must predict experience separately from quarter of birth).
Exercise 5 · Controls in both stages
Why must region dummies that appear in the wage equation also appear in the first stage?
Show solution
Step 1. If region is omitted from the first stage, the first-stage residual contains the part of explained by region.
Step 2. Region is in the second stage, so that residual is correlated with a second-stage regressor; the estimator no longer equals with a valid and becomes inconsistent in general.
Answer: exogenous controls are their own instruments; include them in .
Part 2 · Proofs
Exercise 6 · Two stages give 2SLS
Prove that OLS of on equals .
Show solution
Step 1. OLS: .
Step 2. (Exercise 2).
Step 3. .
Exercise 7 · Just identification
If and is invertible, prove .
Show solution
Step 1. , a product of three square invertible matrices.
Step 2. Its inverse is .
Step 3. Multiply by : everything cancels except .
Exercise 8 · Consistency
Assume (invertible), (rank ) and . Prove .
Show solution
Step 1. Substitute : .
Step 2. Scale by : , invertible by the rank condition.
Step 3. .
Step 4. By Slutsky, the second term vanishes.
Exercise 9 · Why manual second-stage SEs are wrong
Show that the manual residual equals asymptotically (scalar case), so , where is the first-stage error.
Show solution
Step 1. .
Step 2. With , the manual residual is .
Step 3. Its mean square converges to .
Answer: unless , this differs from ; the standard errors are wrong (usually too large, but too small when is large).
Exercise 10 · 2SLS is GMM
Show that minimising gives the 2SLS estimator.
Show solution
Step 1. .
Step 2. Gradient: .
Step 3. Solve: .
Part 3 · Applied
Exercise 11 · Two instruments for schooling
With demeaned data and two orthogonal instruments (a quarter-of-birth indicator and a compulsory-schooling-law indicator), , and (log wages). Compute the two single-instrument IV estimates and the 2SLS estimate, and show 2SLS is their weighted average.
Show solution
Step 1. Single instruments: and .
Step 2. .
Step 3. .
Step 4. .
Step 5. Weights proportional to : and ; .
Answer: 14% per year; the stronger instrument gets four times the weight.
Exercise 12 · A first stage with controls
A study of the effect of class size on test scores instruments class size with a rule that splits classes above 40 pupils (Maimonides' rule), controlling for enrolment. Write the two stages and say what identifies the effect.
Show solution
Step 1. First stage: class size , where is the rule's predicted class size.
Step 2. Second stage: score .
Step 3. Enrolment is in both stages; identification comes from the rule's discontinuous jumps in predicted class size, not from enrolment's smooth effect.
Answer: a 2SLS with an RD flavour (Day 76).
Exercise 13 · Manual standard errors in a simulation
In the simulation below (returns to schooling, true effect 0.08), compare the correct 2SLS standard error with the one from running the second stage by OLS.
Show solution
Step 1. The correct SE uses residuals .
Step 2. The manual SE uses , which adds to every residual (Exercise 9).
Step 3. Here the first-stage error has a large variance relative to , so the manual SE is wrong by a noticeable factor (the code prints both, and the Monte Carlo standard deviation for reference).
Answer: always use the IV routine's standard errors (or compute residuals with the original regressors).
Exercise 14 · The forbidden regression
To estimate the effect of hospital admission (binary) on costs with distance as instrument, an analyst fits a probit of on distance and covariates, then regresses costs on the fitted probabilities. Why is this risky, and what should be done instead?
Show solution
Step 1. 2SLS's consistency relies on the first-stage residual being orthogonal to all second-stage regressors by construction (linear projection).
Step 2. A probit's residual has no such property; if the probit is misspecified, the second stage inherits a correlation between the fitted value and the error.
Step 3. Safe alternatives: linear first stage (2SLS), or use the probit fitted probability as an instrument for in a linear IV.
Answer: the plug-in version is consistent only if the probit is exactly right; using the probit as an instrument stays consistent regardless.
Exercise 15 · Heteroskedasticity-robust standard errors
In an IV regression of firm productivity on management training (instrumented by a subsidy lottery), residual variance is larger for small firms. Write the robust variance estimator and explain why it is preferred.
Show solution
Step 1. with .
Step 2. The homoskedastic formula assumes , false here.
Answer: the sandwich is consistent either way (Day 29); with clustered assignment (lotteries by region), cluster the middle term.
Verify in Python
import numpy as np
# Exercises 1 and 3
z = np.array([1, -1, 1, -1.]); x = np.array([2, 0, 3, -1.]); y = np.array([5, 1, 7, -1.])
print("IV:", z @ y / (z @ x), " first stage:", z @ x / (z @ z))
# Exercise 11 from cross-products
ZZ = np.diag([100., 100.]); Zx = np.array([20., 10.]); Zy = np.array([3., 1.])
print("single IVs:", Zy / Zx, " 2SLS:", round((Zx @ np.linalg.solve(ZZ, Zy)) / (Zx @ np.linalg.solve(ZZ, Zx)), 4))
# 2SLS by matrices, correct versus manual standard errors, with a Monte Carlo check (Exercise 13)
rng = np.random.default_rng(64)
def draw(n=2000):
A = rng.normal(size=n); Z = rng.normal(size=(n, 2))
S = 12 + Z @ np.array([.5, .3]) + 1.5 * A + rng.normal(0, 2, n)
lw = 1 + .08 * S + .1 * A + rng.normal(0, .3, n)
return lw, np.column_stack([np.ones(n), S]), np.column_stack([np.ones(n), Z])
def tsls(y, X, Z):
Xh = Z @ np.linalg.lstsq(Z, X, rcond=None)[0]
b = np.linalg.solve(Xh.T @ Xh, Xh.T @ y)
inv = np.linalg.inv(Xh.T @ Xh)
se_ok = np.sqrt((y - X @ b) @ (y - X @ b) / len(y) * inv[1, 1])
se_manual = np.sqrt((y - Xh @ b) @ (y - Xh @ b) / len(y) * inv[1, 1])
return b[1], se_ok, se_manual
y, X, Z = draw()
b, s1, s2 = tsls(y, X, Z)
print(f"2SLS {b:.4f} correct SE {s1:.4f} manual SE {s2:.4f}")
reps = [tsls(*draw())[0] for _ in range(500)]
print(f"Monte Carlo SD of 2SLS: {np.std(reps):.4f}")
Key takeaways
- : OLS on first-stage fitted values, with exogenous controls in both stages.
- Identification needs as many instruments as endogenous regressors (order) and a full-rank first stage (rank).
- Compute residuals with the actual regressors; never plug nonlinear first-stage predictions into the second stage.
Simulation Lab: Two-Stage Least Squares
Open full screen →Watch both stages, and see why manual second-stage standard errors are wrong.
Stage 1 projects the endogenous regressor on the instrument (left); stage 2 regresses the outcome on the fitted values (right). The slope is 2SLS. Standard errors must use residuals computed with the actual x, not the fitted x̂: compare the two.