Day 19: Inverses, Determinants & Systems of Equations

Theory

Linear systems

A system of $n$ equations in $n$ unknowns is $Ax = b$ . If $A$ has an inverse $A^{-1}$ (with $AA^{-1} = A^{-1}A = I$ ), the unique solution is $x = A^{-1}b$ .

The 2×2 case

A = \begin{pmatrix}a & b\\c & d\end{pmatrix}, \qquad \det A = ad - bc, \qquad A^{-1} = \frac{1}{ad - bc}\begin{pmatrix}d & -b\\-c & a\end{pmatrix}.

Determinants

For 3×3 matrices, expand along the first row (cofactor expansion):

\det A = a_{11}\begin{vmatrix}a_{22} & a_{23}\\a_{32} & a_{33}\end{vmatrix} - a_{12}\begin{vmatrix}a_{21} & a_{23}\\a_{31} & a_{33}\end{vmatrix} + a_{13}\begin{vmatrix}a_{21} & a_{22}\\a_{31} & a_{32}\end{vmatrix}.

Key facts:

$|\det A|$ is the factor by which $A$ scales areas (volumes); the sign says whether it flips orientation.
$\det(AB) = \det A\det B$ , $\det(A^{\top}) = \det A$ , $\det(A^{-1}) = 1/\det A$ .
$A$ is invertible $\iff \det A \ne 0 \iff$ full rank $\iff Ax = 0$ only for $x = 0$ .

Inverse rules

(AB)^{-1} = B^{-1}A^{-1}, \qquad (A^{\top})^{-1} = (A^{-1})^{\top}, \qquad (cA)^{-1} = c^{-1}A^{-1}.

In practice, software never forms $A^{-1}$ to solve a system; it uses elimination (LU or QR factorisations), which is faster and more accurate.

Conditioning

If $\det A$ is close to zero relative to the size of the entries, $A$ is ill-conditioned: tiny changes in $b$ or $A$ produce large changes in $x$ . The condition number (ratio of largest to smallest singular value) measures this; Day 21 relates it to eigenvalues of $X^{\top}X$ .

Part 1 · Warm-up

Exercise 1 · A determinant

Compute $\det\begin{pmatrix}3 & 1\\2 & 4\end{pmatrix}$ .

Show solution

$3 \times 4 - 1 \times 2 = 10$ .

Exercise 2 · An inverse

Find the inverse of the matrix in Exercise 1 and check it.

Show solution

$A^{-1} = \frac{1}{10}\begin{pmatrix}4 & -1\\-2 & 3\end{pmatrix}$ . Check: $AA^{-1} = \frac{1}{10}\begin{pmatrix}12 - 2 & -3 + 3\\8 - 8 & -2 + 12\end{pmatrix} = I$ .

Exercise 3 · Solving a system

Solve $3x + y = 9$ , $2x + 4y = 16$ .

Show solution

Step 1. Write the system as $A(x, y)^{\top} = (9, 16)^{\top}$ with $A^{-1} = \frac{1}{10}\begin{pmatrix}4 & -1\\-2 & 3\end{pmatrix}$ (Exercise 2).

Step 2. $x = \frac{1}{10}(4 \cdot 9 - 1 \cdot 16) = \frac{20}{10} = 2$ .

Step 3. $y = \frac{1}{10}(-2 \cdot 9 + 3 \cdot 16) = \frac{30}{10} = 3$ .

Step 4. Check: $3(2) + 3 = 9$ and $2(2) + 4(3) = 16$ . ✓

Answer: $(x, y) = (2, 3)$ .

Exercise 4 · A 3×3 determinant

Compute $\det\begin{pmatrix}2 & 0 & 1\\1 & 3 & 2\\1 & 1 & 1\end{pmatrix}$ . Is the matrix invertible?

Show solution

$2(3 - 2) - 0 + 1(1 - 3) = 2 - 2 = 0$ . Singular: not invertible. (Column 3 = ½ column 1 + ½ column 2.)

Exercise 5 · Area scaling

$A = \begin{pmatrix}2 & 1\\0 & 3\end{pmatrix}$ maps the unit square to a parallelogram. What is its area?

Show solution

$|\det A| = 6$ . The images of $(1, 0)$ and $(0, 1)$ are $(2, 0)$ and $(1, 3)$ : base 2, height 3.

Part 2 · Proofs

Exercise 6 · Inverse of a product

Prove $(AB)^{-1} = B^{-1}A^{-1}$ for invertible $A$ and $B$ .

Show solution

$(AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AA^{-1} = I$ , and similarly in the other order. Inverses are unique, so this is $(AB)^{-1}$ .

Exercise 7 · Inverse of a transpose

Prove $(A^{\top})^{-1} = (A^{-1})^{\top}$ . Conclude that the inverse of a symmetric matrix is symmetric.

Show solution

Step 1. $A^{\top}(A^{-1})^{\top} = (A^{-1}A)^{\top} = I^{\top} = I$ .

Step 2. If $A = A^{\top}$ , then $A^{-1} = (A^{\top})^{-1} = (A^{-1})^{\top}$ .

Step 3. So $(X^{\top}X)^{-1}$ is symmetric, as a covariance matrix must be.

Exercise 8 · The 2×2 formula

Verify that the 2×2 inverse formula satisfies $AA^{-1} = I$ .

Show solution

$\begin{pmatrix}a & b\\c & d\end{pmatrix}\begin{pmatrix}d & -b\\-c & a\end{pmatrix} = \begin{pmatrix}ad - bc & 0\\0 & ad - bc\end{pmatrix}$ . Dividing by $ad - bc$ gives $I$ ; this is possible only if $ad - bc \ne 0$ .

Exercise 9 · Determinant of the inverse

Using $\det(AB) = \det A\det B$ , prove $\det(A^{-1}) = 1/\det A$ .

Show solution

$1 = \det I = \det(AA^{-1}) = \det A\cdot\det(A^{-1})$ . So $\det A \ne 0$ and $\det(A^{-1}) = 1/\det A$ .

Exercise 10 · Unique, none, or infinitely many

Show that if $A$ is invertible, $Ax = b$ has exactly one solution. Give a singular example with no solution and one with infinitely many.

Show solution

Step 1 (existence). $x = A^{-1}b$ works: $A(A^{-1}b) = b$ .

Step 2 (uniqueness). If $Ax_1 = Ax_2 = b$ , then $A(x_1 - x_2) = 0$ ; multiply by $A^{-1}$ to get $x_1 - x_2 = 0$ .

Step 3 (singular, no solution). $A = \begin{pmatrix}1 & 1\\1 & 1\end{pmatrix}$ , $b = (1, 2)$ : the equations $x + y = 1$ and $x + y = 2$ are parallel lines.

Step 4 (singular, infinitely many). Same $A$ with $b = (1, 1)$ : both equations are the line $x + y = 1$ . $\blacksquare$

Part 3 · Applied

Exercise 11 · Market equilibrium

Demand $Q = 100 - 2P + 0.5Y$ and supply $Q = 20 + 3P$ , with income $Y = 40$ . Write the system as $A(Q, P)^{\top} = b$ and solve it.

Show solution

Step 1. Substitute $Y = 40$ : demand is $Q = 120 - 2P$ , i.e. $Q + 2P = 120$ . Supply is $Q - 3P = 20$ .

Step 2. Matrix form:

\begin{pmatrix}1 & 2\\1 & -3\end{pmatrix}\begin{pmatrix}Q\\P\end{pmatrix} = \begin{pmatrix}120\\20\end{pmatrix}, \qquad \det A = (1)(-3) - (2)(1) = -5.

Step 3. Inverse: $A^{-1} = \frac{1}{-5}\begin{pmatrix}-3 & -2\\-1 & 1\end{pmatrix}$ .

Step 4. $Q = \frac{(-3)(120) + (-2)(20)}{-5} = \frac{-400}{-5} = 80$ and $P = \frac{(-1)(120) + (1)(20)}{-5} = \frac{-100}{-5} = 20$ .

Answer: equilibrium price 20 and quantity 80.

Exercise 12 · Comparative statics

In Exercise 11, find $dP/dY$ and $dQ/dY$ .

Show solution

Step 1. Income enters only the right-hand side: $b = (100 + 0.5Y,\; 20)^{\top}$ , so $db/dY = (0.5, 0)^{\top}$ .

Step 2. $(Q, P)^{\top} = A^{-1}b$ , hence $d(Q, P)^{\top}/dY = A^{-1}(0.5, 0)^{\top}$ .

Step 3. Using $A^{-1}$ from Exercise 11: $\frac{1}{-5}\big((-3)(0.5),\; (-1)(0.5)\big) = \frac{1}{-5}(-1.5, -0.5) = (0.3, 0.1)$ .

Answer: $dQ/dY = 0.3$ and $dP/dY = 0.1$ : a $1 rise in income raises quantity by 0.3 and price by 0.1.

Exercise 13 · An ill-conditioned diet plan

A school-meals programme mixes two foods, $x$ and $y$ (kg per pupil per week), whose nutrient contents are almost identical. It must hit a calorie target and a protein target:

x + y = 2, \qquad x + 1.001\,y = 2.001.

Solve the system. Then suppose the protein target is revised to 2.002 and solve again.

Show solution

Step 1. Subtract the first equation from the second: $0.001\,y = 0.001 \Rightarrow y = 1$ , so $x = 1$ .

Step 2. With the revised target: $0.001\,y = 0.002 \Rightarrow y = 2$ , so $x = 0$ .

Step 3. Why: $\det A = 1.001 - 1 = 0.001$ ; the two lines are almost parallel (condition number ≈ 4,000).

Answer: a 0.05% change in one target switches the plan from 1 kg of each food to 2 kg of one and none of the other. Collinear regressors make $\hat\beta$ fragile in exactly this way.

Exercise 14 · Leontief input–output

Two sectors use each other's output: $A = \begin{pmatrix}0.2 & 0.3\\0.4 & 0.1\end{pmatrix}$ , final demand $d = (100, 50)$ . Gross output solves $x = Ax + d$ . Find $x$ .

Show solution

Step 1. $x = Ax + d \iff (I - A)x = d$ .

Step 2. $I - A = \begin{pmatrix}0.8 & -0.3\\-0.4 & 0.9\end{pmatrix}$ , with determinant $0.8(0.9) - (-0.3)(-0.4) = 0.72 - 0.12 = 0.60$ .

Step 3. $(I - A)^{-1} = \frac{1}{0.6}\begin{pmatrix}0.9 & 0.3\\0.4 & 0.8\end{pmatrix}$ .

Step 4. $x_1 = \frac{0.9(100) + 0.3(50)}{0.6} = \frac{105}{0.6} = 175$ and $x_2 = \frac{0.4(100) + 0.8(50)}{0.6} = \frac{80}{0.6} \approx 133.3$ .

Answer: gross output (175, 133.3): delivering 100 and 50 to final users requires extra production for intermediate use.

Exercise 15 · The IV estimator solves a system

With one instrument $z$ and one regressor $x$ (both centred), the IV condition is $z^{\top}(y - x\beta) = 0$ . Solve for $\beta$ , and evaluate it when $\widehat{\operatorname{Cov}}(z, y) = 0.6$ and $\widehat{\operatorname{Cov}}(z, x) = 0.3$ .

Show solution

Step 1. Expand the condition: $z^{\top}y - z^{\top}x\,\beta = 0$ .

Step 2. Solve: $\hat\beta = (z^{\top}x)^{-1}z^{\top}y$ . Dividing numerator and denominator by $n - 1$ turns these into sample covariances.

Step 3. $\hat\beta = 0.6/0.3 = 2$ .

Answer: 2, the Wald ratio from Day 12. With several instruments and regressors, $\hat\beta = (Z^{\top}X)^{-1}Z^{\top}y$ , and a weak first stage ( $Z^{\top}X$ near singular) is an ill-conditioned system like Exercise 13.

Verify in Python

import numpy as np

A = np.array([[3, 1], [2, 4.]])
print("det =", round(np.linalg.det(A), 6), "\ninverse =\n", np.linalg.inv(A))
print("solution:", np.linalg.solve(A, [9, 16]))
print("3x3 det:", round(np.linalg.det(np.array([[2, 0, 1], [1, 3, 2], [1, 1, 1.]])), 10))

M = np.array([[1, 2], [1, -3.]])
print("equilibrium (Q, P):", np.linalg.solve(M, [120, 20]), " d/dY:", np.linalg.solve(M, [0.5, 0]))

B = np.array([[1, 1], [1, 1.001]])
print("ill-conditioned:", np.linalg.solve(B, [2, 2.001]), np.linalg.solve(B, [2, 2.002]),
      " condition number:", round(np.linalg.cond(B)))

L = np.array([[0.2, 0.3], [0.4, 0.1]])
print("Leontief output:", np.linalg.solve(np.eye(2) - L, [100, 50]).round(2))

Key takeaways

$Ax = b$ has a unique solution iff $\det A \ne 0$ , iff $A$ has full rank.
$|\det A|$ is an area/volume scale factor; a determinant near zero means a nearly flat (fragile) map.
Equilibrium models, input–output models and IV estimators are all linear systems.

Day 19. Inverses, Determinants & Systems of Equations

Theory

Linear systems

The 2×2 case

Determinants

Inverse rules

Conditioning

Part 1 · Warm-up

Exercise 1 · A determinant

Exercise 2 · An inverse

Exercise 3 · Solving a system

Exercise 4 · A 3×3 determinant

Exercise 5 · Area scaling

Part 2 · Proofs

Exercise 6 · Inverse of a product

Exercise 7 · Inverse of a transpose

Exercise 8 · The 2×2 formula

Exercise 9 · Determinant of the inverse

Exercise 10 · Unique, none, or infinitely many

Part 3 · Applied

Exercise 11 · Market equilibrium

Exercise 12 · Comparative statics

Exercise 13 · An ill-conditioned diet plan

Exercise 14 · Leontief input–output

Exercise 15 · The IV estimator solves a system

Verify in Python

Key takeaways

Simulation Lab: Solving Ax = b