All simulators
Collinearity & Rank
Raise the correlation between two regressors and watch the cloud of coefficient estimates stretch along a line.
Day 18: Linear Independence, Span & RankVIF = 1/(1−ρ²)
5.3
sd of β̂₁ (simulated)
0.476
theory ≈ 0.459
corr(β̂₁, β̂₂)
-0.91
≈ −ρ
condition no. of X′X
19.0
(1+ρ)/(1−ρ)
Each dot is (β̂₁, β̂₂) from one simulated sample of y = x₁ + x₂ + noise. As corr(x₁, x₂) → 1 the cloud stretches along a line of slope −1: the data pin down β₁ + β₂ well, but hardly β₁ − β₂.