Day 66: Over-Identification & Instrument Validity

Theory

Over-identifying restrictions

With $l$ instruments and $k$ endogenous regressors ( $l > k$ ), the moment conditions $E[\mathbf z_i\varepsilon_i] = \mathbf 0$ are more than needed. If all instruments are valid, every just-identified subset estimates the same $\beta$ (with a constant effect), and the residual should be unrelated to every instrument.

The Sargan–Hansen J test

Estimate by 2SLS and compute residuals $\hat\varepsilon = \mathbf y - \mathbf X\hat{\boldsymbol\beta}$ .
Regress $\hat\varepsilon$ on all instruments (and controls); record $R^2$ .
$J = nR^2 \sim \chi^2_{l - k}$ under the null that all instruments are valid (homoskedastic errors). Hansen's J, the minimised two-step GMM objective, is the robust version.

What J can and cannot do

If one instrument is invalid and others valid, J has power to detect the disagreement, but it cannot say which instrument is wrong.
If all instruments are invalid in the same way (estimates biased by the same amount), J has no power (Exercise 9).
With heterogeneous effects, valid instruments can identify different LATEs (Day 63), so J can reject even when every instrument is valid.

Is the regressor endogenous?

The Durbin–Wu–Hausman test compares OLS and IV. A convenient regression version is the control function: (1) run the first stage and save the residual $\hat v$ ; (2) regress $Y$ on $X$ , controls and $\hat v$ . The coefficient on $X$ equals 2SLS, and a significant coefficient on $\hat v$ rejects exogeneity of $X$ .

Plausibly exogenous instruments

If $Z$ has a small direct effect $\gamma$ on $Y$ , then $\beta_{IV} = \beta + \gamma/\pi$ . Allowing $\gamma \in [-\delta, \delta]$ , the union of the IV confidence intervals shifted by $\gamma/\pi$ gives a sensitivity interval (Conley, Hansen and Rossi). Weak first stages magnify small violations.

Part 1 · Warm-up

Exercise 1 · Degrees of freedom

A model has 2 endogenous regressors and 5 excluded instruments. What are the degrees of freedom of the J test?

Show solution

Step 1. $l - k = 5 - 2 = 3$ over-identifying restrictions.

Answer: $\chi^2_3$ .

Exercise 2 · Computing J

With $n = 1{,}000$ , regressing the 2SLS residuals on 3 instruments (1 endogenous regressor) gives $R^2 = 0.006$ . Compute J and test at 5% ( $\chi^2_{2, 0.95} = 5.99$ ).

Show solution

Step 1. $J = 1{,}000 \times 0.006 = 6.0$ .

Step 2. $6.0 > 5.99$ : reject (barely).

Answer: the instruments disagree more than chance would explain, at the 5% level.

Exercise 3 · Comparing two just-identified estimates

Two instruments give 0.15 (SE 0.02) and 0.10 (SE 0.04), with approximately independent estimates. Test their equality.

Show solution

Step 1. Difference: 0.05; SE $\sqrt{0.02^2 + 0.04^2} = \sqrt{0.002} \approx 0.0447$ .

Step 2. $z = 0.05/0.0447 \approx 1.12 < 1.96$ .

Answer: not rejected; the instruments are statistically compatible.

Exercise 4 · A control-function test

In the regression of $Y$ on $X$ and the first-stage residual $\hat v$ , the coefficient on $\hat v$ has $t = 2.5$ . What do you conclude?

Show solution

Step 1. $|t| > 1.96$ : reject $H_0$ that $X$ is exogenous.

Answer: OLS and IV differ significantly; use IV (assuming the instrument is valid).

Exercise 5 · Just identified

What is J in a just-identified model, and why?

Show solution

Step 1. With $l = k$ , 2SLS sets $\mathbf Z^\top\hat{\boldsymbol\varepsilon} = \mathbf 0$ exactly (Day 64, Exercise 7).

Step 2. The residuals are orthogonal to all instruments, so $R^2 = 0$ and $J = 0$ .

Answer: zero: just-identified models have nothing to test.

Part 2 · Proofs

Exercise 6 · J = 0 at an exact fit

Prove that in a just-identified model $\mathbf Z^\top(\mathbf y - \mathbf X\hat{\boldsymbol\beta}_{IV}) = \mathbf 0$ .

Show solution

Step 1. $\hat{\boldsymbol\beta}_{IV} = (\mathbf Z^\top\mathbf X)^{-1}\mathbf Z^\top\mathbf y$ .

Step 2. $\mathbf Z^\top\mathbf X\hat{\boldsymbol\beta}_{IV} = \mathbf Z^\top\mathbf y$ .

Step 3. Subtract. $\blacksquare$

Exercise 7 · The control function reproduces 2SLS

Scalar endogenous $x$ , instrument vector $\mathbf Z$ , first stage $x = \mathbf Z\hat\pi + \hat v$ . Prove that in the OLS regression of $y$ on $x$ and $\hat v$ , the coefficient on $x$ equals 2SLS.

Show solution

Step 1. By FWL, the coefficient on $x$ uses $x$ residualised on $\hat v$ .

Step 2. $\hat v$ is orthogonal to $\mathbf Z\hat\pi = \hat x$ , and $x = \hat x + \hat v$ , so residualising $x$ on $\hat v$ gives exactly $\hat x$ .

Step 3. The coefficient is $\hat x^\top y/\hat x^\top\hat x$ , the 2SLS estimator. $\blacksquare$

Exercise 8 · Disagreement is what J detects

With two instruments (one endogenous regressor), show that if both single-instrument estimands are equal, the population over-identifying restriction holds in the sense that some $\beta$ satisfies both moment conditions.

Show solution

Step 1. Moment conditions: $E[z_j(y - \beta x)] = 0$ for $j = 1, 2$ , i.e. $\beta = E[z_j y]/E[z_j x]$ .

Step 2. If both ratios equal the same $b$ , then $\beta = b$ solves both.

Step 3. Conversely, if the ratios differ, no single $\beta$ solves both, and J diverges with $n$ . $\blacksquare$

Exercise 9 · No power against common invalidity

Suppose each instrument has a direct effect proportional to its first stage: $\delta_j = c\,\pi_j$ . Show that every single-instrument estimand equals $\beta + c$ , so J has no power.

Show solution

Step 1. From Day 62, Exercise 10: instrument $j$ gives $\beta + \delta_j/\pi_j$ .

Step 2. $\delta_j/\pi_j = c$ for all $j$ , so all estimands equal $\beta + c$ .

Step 3. By Exercise 8, the moment conditions are jointly satisfied at $\beta + c$ : J converges to $\chi^2_{l-k}$ as under validity. $\blacksquare$

Exercise 10 · Plausibly exogenous bounds

Let $Y = \beta X + \gamma Z + \varepsilon$ with $\gamma \in [-\delta, \delta]$ unknown. Show that the IV estimand is $\beta + \gamma/\pi$ and that $\beta \in [\beta_{IV} - \delta/|\pi|,\; \beta_{IV} + \delta/|\pi|]$ .

Show solution

Step 1. $\operatorname{Cov}(Z, Y) = \beta\operatorname{Cov}(Z, X) + \gamma\operatorname{Var}(Z)$ .

Step 2. Divide by $\operatorname{Cov}(Z, X) = \pi\operatorname{Var}(Z)$ : $\beta_{IV} = \beta + \gamma/\pi$ .

Step 3. $\beta = \beta_{IV} - \gamma/\pi$ , which ranges over the stated interval as $\gamma$ ranges over $[-\delta, \delta]$ . $\blacksquare$

Part 3 · Applied

Exercise 11 · Two instruments for schooling

Day 64's instruments give single-instrument returns of 0.15 (SE 0.02, the quarter-of-birth indicator) and 0.10 (SE 0.04, the schooling-law indicator). Is there evidence of invalidity? What else could explain a difference?

Show solution

Step 1. By Exercise 3, $z \approx 1.12$ : no significant disagreement.

Step 2. Even a significant difference could reflect different LATEs: compulsory-schooling laws affect students at risk of dropping out, while quarter of birth matters for others.

Answer: no evidence against validity; and disagreement would not prove invalidity.

Exercise 12 · One bad instrument in a simulation

In the simulation below, two instruments are valid when the second's direct effect is 0, and the second is invalid when its direct effect is 0.05 or 0.10. How do the 2SLS estimate (truth 0.5) and the J test's rejection rate change?

Show solution

Step 1. Direct effect 0: 2SLS ≈ 0.50 and J rejects about 5% of the time (correct size).

Step 2. Direct effect 0.05: 2SLS ≈ 0.58; J rejects about 22%.

Step 3. Direct effect 0.10: 2SLS ≈ 0.67; J rejects about 65%.

Answer: J detects the bad instrument only part of the time, while the bias is already substantial: failing to reject is weak evidence of validity.

Exercise 13 · Is insurance endogenous?

Using the Oregon-style lottery (Day 62), the control-function regression of ER visits on insurance and the first-stage residual gives a residual coefficient of −0.3 (SE 0.1). Interpret.

Show solution

Step 1. $t = -3$ : insurance is endogenous.

Step 2. The negative sign means that, holding insurance fixed, unobservables that raise insurance take-up lower ER visits (for example, people who sign up for insurance are healthier or better organised).

Answer: OLS of ER visits on insurance would understate the effect; IV is preferred.

Exercise 14 · Plausibly exogenous college proximity

Distance to college might affect wages directly (urban labour markets). Allow the direct effect on log wages to be between −0.01 and 0.01, with first stage 0.32 and IV estimate 0.131 (95% CI [0.05, 0.21]). Compute the sensitivity interval.

Show solution

Step 1. Shift: $\delta/\pi = 0.01/0.32 \approx 0.031$ .

Step 2. Union of shifted intervals: $[0.05 - 0.031,\; 0.21 + 0.031] = [0.019, 0.241]$ .

Answer: the return stays positive, but the range widens considerably; a weaker first stage would widen it more.

Exercise 15 · Rejection with valid instruments

A study of the effect of child care subsidies on maternal employment uses two valid instruments: a lottery for subsidised places and the birth date cutoff for kindergarten eligibility. J rejects strongly. Give an explanation consistent with validity and what to report.

Show solution

Step 1. The lottery's compliers are mothers who applied (motivated to work); the birth-date compliers are all mothers of children near the cutoff.

Step 2. With heterogeneous effects, the two LATEs differ, so a constant-effect J test rejects.

Answer: report the two estimates separately, describe each set of compliers (Day 63), and avoid pooling them into one 2SLS number.

Verify in Python

import numpy as np
from scipy import stats

rng = np.random.default_rng(66); n = 1000
def tsls(y, X, Z):
    Xh = Z @ np.linalg.lstsq(Z, X, rcond=None)[0]
    return np.linalg.solve(Xh.T @ Xh, Xh.T @ y)

print("direct effect  J rejection  mean 2SLS (truth 0.5)")
for d in (0, .05, .10):
    rej, est = 0, []
    for _ in range(1000):
        Z = rng.normal(size=(n, 2)); u = rng.normal(size=n); v = .5 * u + rng.normal(size=n)
        x = Z @ [.3, .3] + v; y = 1 + .5 * x + d * Z[:, 1] + u
        X = np.column_stack([np.ones(n), x]); ZZ = np.column_stack([np.ones(n), Z])
        b = tsls(y, X, ZZ); r = y - X @ b
        g = np.linalg.lstsq(ZZ, r, rcond=None)[0]
        R2 = 1 - ((r - ZZ @ g) @ (r - ZZ @ g)) / ((r - r.mean()) @ (r - r.mean()))
        rej += n * R2 > stats.chi2.ppf(.95, 1); est.append(b[1])
    print(f"{d:13.2f}  {rej / 1000:11.3f}  {np.mean(est):.3f}")

# Exercise 7: the control function reproduces 2SLS
Z = rng.normal(size=(n, 2)); u = rng.normal(size=n); x = Z @ [.3, .3] + .5 * u + rng.normal(size=n); y = .5 * x + u
ZZ = np.column_stack([np.ones(n), Z]); vhat = x - ZZ @ np.linalg.lstsq(ZZ, x, rcond=None)[0]
cf = np.linalg.lstsq(np.column_stack([np.ones(n), x, vhat]), y, rcond=None)[0]
print("control function:", cf[1].round(5), " 2SLS:", tsls(y, np.column_stack([np.ones(n), x]), ZZ)[1].round(5), " residual coef:", cf[2].round(3))

# Exercises 2, 3 and 14
print("J:", 1000 * .006, "crit", round(stats.chi2.ppf(.95, 2), 2), "| z:", round(.05 / np.sqrt(.02**2 + .04**2), 2),
      "| sensitivity interval:", np.round([.05 - .01 / .32, .21 + .01 / .32], 3))

Key takeaways

With extra instruments, J = nR² tests whether the instruments agree; it cannot identify which is wrong and has no power when all are wrong in the same way.
The control-function regression reproduces 2SLS and tests whether OLS is biased.
With heterogeneous effects, disagreement can reflect different LATEs; sensitivity intervals show how much a small exclusion violation matters.

Day 66. Over-Identification & Instrument Validity

Theory

Over-identifying restrictions

The Sargan–Hansen J test

What J can and cannot do

Is the regressor endogenous?

Plausibly exogenous instruments

Part 1 · Warm-up

Exercise 1 · Degrees of freedom

Exercise 2 · Computing J

Exercise 3 · Comparing two just-identified estimates

Exercise 4 · A control-function test

Exercise 5 · Just identified

Part 2 · Proofs

Exercise 6 · J = 0 at an exact fit

Exercise 7 · The control function reproduces 2SLS

Exercise 8 · Disagreement is what J detects

Exercise 9 · No power against common invalidity

Exercise 10 · Plausibly exogenous bounds

Part 3 · Applied

Exercise 11 · Two instruments for schooling

Exercise 12 · One bad instrument in a simulation

Exercise 13 · Is insurance endogenous?

Exercise 14 · Plausibly exogenous college proximity

Exercise 15 · Rejection with valid instruments

Verify in Python

Key takeaways

Simulation Lab: Over-Identification & the J Test