Day 65: Weak Instruments & Robust Inference · Math for Causal Inference

Theory

First-stage strength

In the first stage $X = \pi^\top\mathbf Z + v$ with $l$ excluded instruments, strength is measured by the concentration parameter $\mu^2 = \pi^\top\mathbf Z^\top\mathbf Z\pi/\sigma_v^2$ , estimated through the first-stage F statistic for $H_0: \pi = 0$ . Roughly $E[F] \approx 1 + \mu^2/l$ . With one instrument, $F = t^2$ for the first-stage coefficient.

Bias towards OLS

With weak instruments, 2SLS is biased in the direction of OLS. A useful approximation (many instruments, homoskedastic errors):

\frac{E[\hat\beta_{2SLS}] - \beta}{\operatorname{plim}\hat\beta_{OLS} - \beta} \approx \frac{1}{F}.

When the instruments are irrelevant ( $\pi = 0$ ), 2SLS is centred on the OLS probability limit: weak IV "finds" the OLS answer. Bound, Jaeger and Baker (1995) reproduced quarter-of-birth results with randomly generated instruments.

Bad inference

The 2SLS t-statistic is not approximately normal when $F$ is small: Wald confidence intervals ( $\hat\beta \pm 1.96\,\operatorname{SE}$ ) cover less than 95%, sometimes far less when endogeneity is strong. The rule of thumb $F > 10$ (Staiger and Stock; Stock and Yogo) was designed to keep bias below about 10% of OLS bias. For a just-identified model, Lee, McCrary, Moreira and Porter (2022) show the usual t-test has correct 5% size only if $F > 104.7$ ; for smaller $F$ the critical value must be raised (their tF procedure).

The Anderson–Rubin test

To test $H_0: \beta = \beta_0$ , regress $Y - \beta_0X$ on the instruments (and controls) and test that all instrument coefficients are zero. Under $H_0$ , $Y - \beta_0X = \varepsilon$ , which is unrelated to $\mathbf Z$ , so the test has correct size whatever the first-stage strength. Collecting all $\beta_0$ not rejected gives an AR confidence set, which can be:

a bounded interval (strong enough first stage);
the union of two rays, or the whole real line (weak first stage): the data are honestly uninformative.

Practice

Report the first stage and its F (robust or effective F with heteroskedastic or clustered errors), the reduced form (an unbiased intention-to-treat estimate whatever the first stage), and AR or tF confidence sets when $F$ is not large.

Part 1 · Warm-up

Exercise 1 · F from t

A first-stage coefficient has t-statistic 3. What is the first-stage F with one instrument? Is the instrument "strong" by the F > 10 rule?

Show solution

Step 1. $F = t^2 = 9$ .

Answer: 9, just below 10: weak by the rule of thumb, and far below the 104.7 needed for standard t-tests.

Exercise 2 · Bias towards OLS

OLS overstates a return by 0.05. With first-stage F = 5, roughly how biased is 2SLS?

Show solution

Step 1. Relative bias $\approx 1/F = 0.2$ .

Step 2. 2SLS bias $\approx 0.2 \times 0.05 = 0.01$ , in the same direction as OLS.

Exercise 3 · An Anderson–Rubin test

Reduced form 0.06, first stage 0.4. For each $\beta_0$ , the coefficient on $Z$ when regressing $Y - \beta_0X$ on $Z$ is $0.06 - 0.4\beta_0$ ; take its standard error as 0.02 near the estimate. Is $\beta_0 = 0$ rejected? Find the AR 95% interval.

Show solution

Step 1. $\beta_0 = 0$ : coefficient 0.06, t = 3: rejected.

Step 2. Not rejected when $|0.06 - 0.4\beta_0| \le 1.96 \times 0.02 = 0.0392$ .

Step 3. $0.0208 \le 0.4\beta_0 \le 0.0992$ , so $\beta_0 \in [0.052, 0.248]$ .

Answer: reject zero; AR interval [0.052, 0.248] around the IV estimate 0.15.

Exercise 4 · Which statistic to report?

A paper with one instrument reports first-stage F = 25 and a 2SLS 95% interval of [0.02, 0.30]. What would you ask for?

Show solution

Step 1. F = 25 passes the F > 10 rule but is below 104.7, so the t-based interval may under-cover.

Answer: an AR confidence set (or tF-adjusted interval) and the reduced form with its own standard error.

Exercise 5 · Irrelevant instruments

What does 2SLS estimate if the instrument is completely irrelevant ( $\pi = 0$ )?

Show solution

Step 1. There is no exogenous variation in $X$ to use.

Answer: 2SLS does not converge to $\beta$ ; its distribution is centred near the OLS probability limit and does not shrink with $n$ .

Part 2 · Proofs

Exercise 6 · AR has correct size

Assume $Y = \beta X + \varepsilon$ with $\varepsilon \sim N(0, \sigma^2)$ independent of $Z$ (fixed). Prove that under $H_0: \beta = \beta_0$ the AR statistic has an exact F distribution whatever $\pi$ .

Show solution

Step 1. Under $H_0$ , $Y - \beta_0X = \varepsilon$ .

Step 2. Regressing $\varepsilon$ on $Z$ and testing the coefficients is a classical F test with normal errors independent of the regressors.

Step 3. Its null distribution is $F_{l, n - l - 1}$ , which does not involve $X$ or $\pi$ . $\blacksquare$

Exercise 7 · Irrelevant instruments centre on OLS

Scalar case, no constant: $X = v$ ( $\pi = 0$ ), $Y = \beta X + \varepsilon$ , with $(\varepsilon, v)$ jointly normal, $\operatorname{Cov}(\varepsilon, v) = \sigma_{\varepsilon v}$ , and $Z$ independent of both. Show that $\hat\beta_{IV} - \beta = \frac{\mathbf z^\top\boldsymbol\varepsilon}{\mathbf z^\top\mathbf v}$ and that, conditional on $\mathbf z^\top\mathbf v$ , its mean is $\sigma_{\varepsilon v}/\sigma_v^2$ , the OLS bias.

Show solution

Step 1. $\hat\beta_{IV} = \mathbf z^\top\mathbf y/\mathbf z^\top\mathbf x = \beta + \mathbf z^\top\boldsymbol\varepsilon/\mathbf z^\top\mathbf v$ .

Step 2. Decompose $\varepsilon = (\sigma_{\varepsilon v}/\sigma_v^2)v + w$ , with $w$ independent of $v$ and $Z$ .

Step 3. $\mathbf z^\top\boldsymbol\varepsilon/\mathbf z^\top\mathbf v = \sigma_{\varepsilon v}/\sigma_v^2 + \mathbf z^\top\mathbf w/\mathbf z^\top\mathbf v$ .

Step 4. Given $\mathbf z^\top\mathbf v$ (and $\mathbf z$ ), $\mathbf z^\top\mathbf w$ has mean 0. $\blacksquare$ So the estimator is centred at $\beta + \sigma_{\varepsilon v}/\sigma_v^2 = \operatorname{plim}\hat\beta_{OLS}$ , with heavy tails from the random denominator.

Exercise 8 · When AR sets are unbounded

Just-identified case: let the reduced form be $\hat\rho$ , the first stage $\hat\pi$ , with variances $s_\rho^2, s_\pi^2$ and covariance 0. The AR set is $\{\beta: (\hat\rho - \beta\hat\pi)^2 \le c\,(s_\rho^2 + \beta^2 s_\pi^2)\}$ with $c = 1.96^2$ . Show that it is unbounded when $\hat\pi^2 < c\,s_\pi^2$ , i.e. when the first stage is not significant at 5%.

Show solution

Step 1. Rearrange: $(\hat\pi^2 - c\,s_\pi^2)\beta^2 - 2\hat\rho\hat\pi\beta + (\hat\rho^2 - c\,s_\rho^2) \le 0$ .

Step 2. If the leading coefficient $\hat\pi^2 - c\,s_\pi^2 < 0$ , the quadratic tends to $-\infty$ as $|\beta| \to \infty$ , so all large $|\beta|$ satisfy the inequality.

Step 3. The set therefore contains both tails: unbounded (two rays, or the whole line). $\blacksquare$

Exercise 9 · F equals t squared

With one instrument, prove that the first-stage F statistic equals the square of the first-stage t statistic.

Show solution

Step 1. The F test of one restriction is $F = (\text{restricted SSR} - \text{unrestricted SSR})/(\text{unrestricted SSR}/(n - k))$ .

Step 2. For a single coefficient, the numerator equals $\hat\pi^2\sum\tilde z_i^2$ (FWL), and $\hat\sigma_v^2 = \text{SSR}/(n - k)$ .

Step 3. So $F = \hat\pi^2\sum\tilde z_i^2/\hat\sigma_v^2 = \big(\hat\pi/\operatorname{SE}(\hat\pi)\big)^2 = t^2$ . $\blacksquare$

Exercise 10 · The reduced form is always informative

Explain why the reduced form $\hat\rho$ is a valid estimate of a causal effect (of the instrument) whatever the first-stage strength, and how its sign relates to that of $\beta$ .

Show solution

Step 1. $Z$ is as good as random, so the regression of $Y$ on $Z$ estimates the causal effect of $Z$ (an ITT), with standard inference.

Step 2. Under exclusion and a constant effect, $\rho = \beta\pi$ ; if the sign of $\pi$ is known, the sign of $\rho$ is the sign of $\beta$ .

Answer: testing $\rho = 0$ is a weak-instrument-robust test of $\beta = 0$ (it is the AR test at $\beta_0 = 0$ ). $\blacksquare$

Part 3 · Applied

Exercise 11 · Quarter of birth with many instruments

Using 180 quarter-of-birth × state × year interactions as instruments, a study of returns to schooling gets a first-stage F of 2 and a 2SLS estimate of 0.061; OLS is 0.063. With 3 quarter dummies, F = 30 and 2SLS is 0.081. Which estimate is more credible?

Show solution

Step 1. With F = 2, the relative bias towards OLS is about 50%: the 180-instrument estimate is pulled almost to OLS.

Step 2. The 3-instrument estimate has a stronger first stage, and less bias.

Answer: the 3-instrument estimate (0.081); many weak instruments manufacture an estimate close to OLS.

Exercise 12 · Coverage in a simulation

In the Monte Carlo below (one instrument, strong endogeneity, true $\beta = 1$ ), what happens to the median 2SLS estimate and to the coverage of the Wald and AR intervals as the first-stage F rises from about 1 to about 46?

Show solution

Step 1. F ≈ 1: the median 2SLS estimate is near the OLS limit (1.8); Wald coverage about 88%; AR 95%.

Step 2. F ≈ 11: median close to 1, but Wald coverage still only about 93%.

Step 3. F ≈ 46: Wald coverage about 95%; AR stays at 95% throughout.

Answer: F > 10 removes most bias but not all over-rejection; AR is valid everywhere.

Exercise 13 · Distance to a specialty hospital

Distance to a cardiac-care hospital instruments for receiving catheterisation (an invasive heart procedure) in a study of survival. The first-stage t is 1.5. What will the AR confidence set look like, and what can still be reported?

Show solution

Step 1. $t^2 = 2.25 < 3.84$ : the first stage is not significant at 5%.

Step 2. By Exercise 8, the AR set is unbounded.

Step 3. The reduced form (effect of distance on survival) can still be reported with standard inference.

Answer: the data cannot bound the effect of catheterisation; say so, report the reduced form, and look for a stronger instrument.

Exercise 14 · Judge leniency designs

Defendants are randomly assigned to judges who differ in leniency; leniency instruments for incarceration. With 200 judges, the first stage uses 200 dummies. What weak-instrument problem arises, and what is the usual fix?

Show solution

Step 1. Many instruments, each estimated from few cases: 2SLS is biased towards OLS (many-instrument bias).

Step 2. Fix: construct a single leave-one-out leniency measure for each judge (the average incarceration rate of the judge's other cases) and use it as one instrument.

Answer: one strong instrument (high F) instead of 200 weak ones; the leave-one-out construction avoids the mechanical correlation with the defendant's own outcome.

Exercise 15 · A reporting checklist

Write a weak-instrument reporting checklist for an IV paper on the effect of microcredit on business profits instrumented by branch openings.

Show solution

Step 1. First stage: coefficient, robust or effective F, and a plot of branch openings against credit take-up.

Step 2. Reduced form: effect of branch openings on profits with its confidence interval.

Step 3. 2SLS estimate with AR (or tF) confidence set, and a note on whether it is bounded.

Step 4. Comparison with OLS and, if over-identified, with single-instrument estimates (Day 66).

Verify in Python

import numpy as np

rng = np.random.default_rng(65); n = 500
print(" pi    mean F   median 2SLS   Wald coverage   AR coverage")
for pi in (0.0, 0.07, 0.14, 0.30):
    F, est, cw, ca = [], [], 0, 0
    for _ in range(2000):
        z = rng.normal(size=n); e = rng.normal(size=n); v = .8 * e + .6 * rng.normal(size=n)
        x = pi * z + v; y = 1.0 * x + e                       # true beta = 1, OLS limit 1.8
        z, x, y = z - z.mean(), x - x.mean(), y - y.mean()
        g = z @ x / (z @ z); r = x - g * z
        F.append(g**2 * (z @ z) / (r @ r / (n - 2)))
        b = z @ y / (z @ x); u = y - b * x
        se = np.sqrt(u @ u / (n - 2) * (z @ z)) / abs(z @ x)
        cw += abs(b - 1) / se < 1.96; est.append(b)
        w = y - 1.0 * x; h = z @ w / (z @ z); rr = w - h * z     # Anderson-Rubin at beta0 = 1
        ca += abs(h / np.sqrt(rr @ rr / (n - 2) / (z @ z))) < 1.96
    print(f"{pi:.2f}   {np.mean(F):6.1f}   {np.median(est):10.3f}   {cw / 2000:12.3f}   {ca / 2000:10.3f}")

# Exercise 3: AR interval
print("AR interval:", np.round([(0.06 - 1.96 * .02) / .4, (0.06 + 1.96 * .02) / .4], 3))

Key takeaways

Weak instruments bias 2SLS towards OLS (roughly by a factor 1/F) and distort t-based inference.
Report the first-stage F, the reduced form, and Anderson–Rubin (or tF) confidence sets; F > 10 limits bias but does not make t-tests reliable.
AR sets are valid for any strength and become unbounded when the first stage is not significant: an honest "we don't know".

Day 65. Weak Instruments & Robust Inference

Theory

First-stage strength

Bias towards OLS

Bad inference

The Anderson–Rubin test

Practice

Part 1 · Warm-up

Exercise 1 · F from t

Exercise 2 · Bias towards OLS

Exercise 3 · An Anderson–Rubin test

Exercise 4 · Which statistic to report?

Exercise 5 · Irrelevant instruments

Part 2 · Proofs

Exercise 6 · AR has correct size

Exercise 7 · Irrelevant instruments centre on OLS

Exercise 8 · When AR sets are unbounded

Exercise 9 · F equals t squared

Exercise 10 · The reduced form is always informative

Part 3 · Applied

Exercise 11 · Quarter of birth with many instruments

Exercise 12 · Coverage in a simulation

Exercise 13 · Distance to a specialty hospital

Exercise 14 · Judge leniency designs

Exercise 15 · A reporting checklist

Verify in Python

Key takeaways

Simulation Lab: Weak Instruments