Linear Algebra & Optimization

Data as vectors, and the dot product as the engine behind means, covariances and correlations.

Multiplying matrices, transposing products, and the cross-product matrices X′X and X′y that every regression is built from.

When columns carry the same information, coefficients are not identified. Rank is how you tell.

Solving Ax = b, what the determinant measures, and why nearly singular systems are fragile.

Least squares is an orthogonal projection. Fitted values, residuals, R² and degrees of freedom all follow from that picture.

The directions a matrix only stretches, and what they reveal about variances, collinearity and stability.

Random vectors, their covariance matrices, and the rules for the variance of any linear combination.

Rates of change, marginal effects and elasticities, and local polynomial approximations of any smooth function.

Differentiating with respect to a whole vector at once, and deriving the OLS, ridge, WLS and logit first-order conditions.

First- and second-order conditions, why convexity makes optimisation easy, and the two algorithms behind every estimator.

Optimising subject to budgets and adding-up constraints, the shadow price, and the experimental-design and weighting problems that use it.

The linear model, the OLS estimator, its mean and variance, s², standard errors and the Gauss–Markov theorem, all in a few lines of matrix algebra.

What "controlling for" really does. Every multiple-regression coefficient is a simple regression on residualised data.

Large-sample theory for OLS without normality or homoskedasticity, and the heteroskedasticity- and cluster-robust standard errors applied work relies on.

One framework for OLS, IV, 2SLS and MLE: choose moment conditions, minimise a quadratic form, and solve it numerically.