{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 19: Inverses, Determinants & Systems of Equations\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "Solving Ax = b, what the determinant measures, and why nearly singular systems are fragile.\n", "\n", "**Learning outcomes**\n", "\n", "- Compute 2×2 and 3×3 determinants and the inverse of a 2×2 matrix.\n", "- Prove the main inverse rules and connect a zero determinant to rank deficiency.\n", "- Solve market-equilibrium and input–output models as linear systems and do comparative statics.\n", "- Explain ill-conditioning and why it makes estimates sensitive to small data changes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### Linear systems\n", "\n", "A system of $n$ equations in $n$ unknowns is $Ax = b$. If $A$ has an **inverse** $A^{-1}$ (with $AA^{-1} = A^{-1}A = I$), the unique solution is $x = A^{-1}b$.\n", "\n", "### The 2×2 case\n", "\n", "$$\n", "A = \\begin{pmatrix}a & b\\\\c & d\\end{pmatrix}, \\qquad \\det A = ad - bc, \\qquad A^{-1} = \\frac{1}{ad - bc}\\begin{pmatrix}d & -b\\\\-c & a\\end{pmatrix}.\n", "$$\n", "\n", "### Determinants\n", "\n", "For 3×3 matrices, expand along the first row (cofactor expansion):\n", "\n", "$$\n", "\\det A = a_{11}\\begin{vmatrix}a_{22} & a_{23}\\\\a_{32} & a_{33}\\end{vmatrix} - a_{12}\\begin{vmatrix}a_{21} & a_{23}\\\\a_{31} & a_{33}\\end{vmatrix} + a_{13}\\begin{vmatrix}a_{21} & a_{22}\\\\a_{31} & a_{32}\\end{vmatrix}.\n", "$$\n", "\n", "Key facts:\n", "\n", "- $|\\det A|$ is the factor by which $A$ scales areas (volumes); the sign says whether it flips orientation.\n", "- $\\det(AB) = \\det A\\det B$, $\\det(A^{\\top}) = \\det A$, $\\det(A^{-1}) = 1/\\det A$.\n", "- $A$ is invertible $\\iff \\det A \\ne 0 \\iff$ full rank $\\iff Ax = 0$ only for $x = 0$.\n", "\n", "### Inverse rules\n", "\n", "$$\n", "(AB)^{-1} = B^{-1}A^{-1}, \\qquad (A^{\\top})^{-1} = (A^{-1})^{\\top}, \\qquad (cA)^{-1} = c^{-1}A^{-1}.\n", "$$\n", "\n", "In practice, software never forms $A^{-1}$ to solve a system; it uses elimination (LU or QR factorisations), which is faster and more accurate.\n", "\n", "### Conditioning\n", "\n", "If $\\det A$ is close to zero relative to the size of the entries, $A$ is **ill-conditioned**: tiny changes in $b$ or $A$ produce large changes in $x$. The **condition number** (ratio of largest to smallest singular value) measures this; Day 21 relates it to eigenvalues of $X^{\\top}X$.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · A determinant\n", "\n", "Compute $\\det\\begin{pmatrix}3 & 1\\\\2 & 4\\end{pmatrix}$.\n", "\n", "
Show solution\n", "\n", "$3 \\times 4 - 1 \\times 2 = 10$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · An inverse\n", "\n", "Find the inverse of the matrix in Exercise 1 and check it.\n", "\n", "
Show solution\n", "\n", "$A^{-1} = \\frac{1}{10}\\begin{pmatrix}4 & -1\\\\-2 & 3\\end{pmatrix}$. Check: $AA^{-1} = \\frac{1}{10}\\begin{pmatrix}12 - 2 & -3 + 3\\\\8 - 8 & -2 + 12\\end{pmatrix} = I$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · Solving a system\n", "\n", "Solve $3x + y = 9$, $2x + 4y = 16$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Write the system as $A(x, y)^{\\top} = (9, 16)^{\\top}$ with $A^{-1} = \\frac{1}{10}\\begin{pmatrix}4 & -1\\\\-2 & 3\\end{pmatrix}$ (Exercise 2).\n", "\n", "**Step 2.** $x = \\frac{1}{10}(4 \\cdot 9 - 1 \\cdot 16) = \\frac{20}{10} = 2$.\n", "\n", "**Step 3.** $y = \\frac{1}{10}(-2 \\cdot 9 + 3 \\cdot 16) = \\frac{30}{10} = 3$.\n", "\n", "**Step 4.** Check: $3(2) + 3 = 9$ and $2(2) + 4(3) = 16$. ✓\n", "\n", "**Answer:** $(x, y) = (2, 3)$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · A 3×3 determinant\n", "\n", "Compute $\\det\\begin{pmatrix}2 & 0 & 1\\\\1 & 3 & 2\\\\1 & 1 & 1\\end{pmatrix}$. Is the matrix invertible?\n", "\n", "
Show solution\n", "\n", "$2(3 - 2) - 0 + 1(1 - 3) = 2 - 2 = 0$. Singular: not invertible. (Column 3 = ½ column 1 + ½ column 2.)\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · Area scaling\n", "\n", "$A = \\begin{pmatrix}2 & 1\\\\0 & 3\\end{pmatrix}$ maps the unit square to a parallelogram. What is its area?\n", "\n", "
Show solution\n", "\n", "$|\\det A| = 6$. The images of $(1, 0)$ and $(0, 1)$ are $(2, 0)$ and $(1, 3)$: base 2, height 3.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · Inverse of a product\n", "\n", "Prove $(AB)^{-1} = B^{-1}A^{-1}$ for invertible $A$ and $B$.\n", "\n", "
Show solution\n", "\n", "$(AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AA^{-1} = I$, and similarly in the other order. Inverses are unique, so this is $(AB)^{-1}$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · Inverse of a transpose\n", "\n", "Prove $(A^{\\top})^{-1} = (A^{-1})^{\\top}$. Conclude that the inverse of a symmetric matrix is symmetric.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $A^{\\top}(A^{-1})^{\\top} = (A^{-1}A)^{\\top} = I^{\\top} = I$.\n", "\n", "**Step 2.** If $A = A^{\\top}$, then $A^{-1} = (A^{\\top})^{-1} = (A^{-1})^{\\top}$.\n", "\n", "**Step 3.** So $(X^{\\top}X)^{-1}$ is symmetric, as a covariance matrix must be.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · The 2×2 formula\n", "\n", "Verify that the 2×2 inverse formula satisfies $AA^{-1} = I$.\n", "\n", "
Show solution\n", "\n", "$\\begin{pmatrix}a & b\\\\c & d\\end{pmatrix}\\begin{pmatrix}d & -b\\\\-c & a\\end{pmatrix} = \\begin{pmatrix}ad - bc & 0\\\\0 & ad - bc\\end{pmatrix}$. Dividing by $ad - bc$ gives $I$; this is possible only if $ad - bc \\ne 0$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Determinant of the inverse\n", "\n", "Using $\\det(AB) = \\det A\\det B$, prove $\\det(A^{-1}) = 1/\\det A$.\n", "\n", "
Show solution\n", "\n", "$1 = \\det I = \\det(AA^{-1}) = \\det A\\cdot\\det(A^{-1})$. So $\\det A \\ne 0$ and $\\det(A^{-1}) = 1/\\det A$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Unique, none, or infinitely many\n", "\n", "Show that if $A$ is invertible, $Ax = b$ has exactly one solution. Give a singular example with no solution and one with infinitely many.\n", "\n", "
Show solution\n", "\n", "**Step 1 (existence).** $x = A^{-1}b$ works: $A(A^{-1}b) = b$.\n", "\n", "**Step 2 (uniqueness).** If $Ax_1 = Ax_2 = b$, then $A(x_1 - x_2) = 0$; multiply by $A^{-1}$ to get $x_1 - x_2 = 0$.\n", "\n", "**Step 3 (singular, no solution).** $A = \\begin{pmatrix}1 & 1\\\\1 & 1\\end{pmatrix}$, $b = (1, 2)$: the equations $x + y = 1$ and $x + y = 2$ are parallel lines.\n", "\n", "**Step 4 (singular, infinitely many).** Same $A$ with $b = (1, 1)$: both equations are the line $x + y = 1$. $\\blacksquare$\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Market equilibrium\n", "\n", "Demand $Q = 100 - 2P + 0.5Y$ and supply $Q = 20 + 3P$, with income $Y = 40$. Write the system as $A(Q, P)^{\\top} = b$ and solve it.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Substitute $Y = 40$: demand is $Q = 120 - 2P$, i.e. $Q + 2P = 120$. Supply is $Q - 3P = 20$.\n", "\n", "**Step 2.** Matrix form:\n", "\n", "$$\n", "\\begin{pmatrix}1 & 2\\\\1 & -3\\end{pmatrix}\\begin{pmatrix}Q\\\\P\\end{pmatrix} = \\begin{pmatrix}120\\\\20\\end{pmatrix}, \\qquad \\det A = (1)(-3) - (2)(1) = -5.\n", "$$\n", "\n", "**Step 3.** Inverse: $A^{-1} = \\frac{1}{-5}\\begin{pmatrix}-3 & -2\\\\-1 & 1\\end{pmatrix}$.\n", "\n", "**Step 4.** $Q = \\frac{(-3)(120) + (-2)(20)}{-5} = \\frac{-400}{-5} = 80$ and $P = \\frac{(-1)(120) + (1)(20)}{-5} = \\frac{-100}{-5} = 20$.\n", "\n", "**Answer:** equilibrium price 20 and quantity 80.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Comparative statics\n", "\n", "In Exercise 11, find $dP/dY$ and $dQ/dY$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Income enters only the right-hand side: $b = (100 + 0.5Y,\\; 20)^{\\top}$, so $db/dY = (0.5, 0)^{\\top}$.\n", "\n", "**Step 2.** $(Q, P)^{\\top} = A^{-1}b$, hence $d(Q, P)^{\\top}/dY = A^{-1}(0.5, 0)^{\\top}$.\n", "\n", "**Step 3.** Using $A^{-1}$ from Exercise 11: $\\frac{1}{-5}\\big((-3)(0.5),\\; (-1)(0.5)\\big) = \\frac{1}{-5}(-1.5, -0.5) = (0.3, 0.1)$.\n", "\n", "**Answer:** $dQ/dY = 0.3$ and $dP/dY = 0.1$: a \\$1 rise in income raises quantity by 0.3 and price by 0.1.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · An ill-conditioned diet plan\n", "\n", "A school-meals programme mixes two foods, $x$ and $y$ (kg per pupil per week), whose nutrient contents are almost identical. It must hit a calorie target and a protein target:\n", "\n", "$$\n", "x + y = 2, \\qquad x + 1.001\\,y = 2.001.\n", "$$\n", "\n", "Solve the system. Then suppose the protein target is revised to 2.002 and solve again.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Subtract the first equation from the second: $0.001\\,y = 0.001 \\Rightarrow y = 1$, so $x = 1$.\n", "\n", "**Step 2.** With the revised target: $0.001\\,y = 0.002 \\Rightarrow y = 2$, so $x = 0$.\n", "\n", "**Step 3.** Why: $\\det A = 1.001 - 1 = 0.001$; the two lines are almost parallel (condition number ≈ 4,000).\n", "\n", "**Answer:** a 0.05% change in one target switches the plan from 1 kg of each food to 2 kg of one and none of the other. Collinear regressors make $\\hat\\beta$ fragile in exactly this way.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Leontief input–output\n", "\n", "Two sectors use each other's output: $A = \\begin{pmatrix}0.2 & 0.3\\\\0.4 & 0.1\\end{pmatrix}$, final demand $d = (100, 50)$. Gross output solves $x = Ax + d$. Find $x$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $x = Ax + d \\iff (I - A)x = d$.\n", "\n", "**Step 2.** $I - A = \\begin{pmatrix}0.8 & -0.3\\\\-0.4 & 0.9\\end{pmatrix}$, with determinant $0.8(0.9) - (-0.3)(-0.4) = 0.72 - 0.12 = 0.60$.\n", "\n", "**Step 3.** $(I - A)^{-1} = \\frac{1}{0.6}\\begin{pmatrix}0.9 & 0.3\\\\0.4 & 0.8\\end{pmatrix}$.\n", "\n", "**Step 4.** $x_1 = \\frac{0.9(100) + 0.3(50)}{0.6} = \\frac{105}{0.6} = 175$ and $x_2 = \\frac{0.4(100) + 0.8(50)}{0.6} = \\frac{80}{0.6} \\approx 133.3$.\n", "\n", "**Answer:** gross output (175, 133.3): delivering 100 and 50 to final users requires extra production for intermediate use.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · The IV estimator solves a system\n", "\n", "With one instrument $z$ and one regressor $x$ (both centred), the IV condition is $z^{\\top}(y - x\\beta) = 0$. Solve for $\\beta$, and evaluate it when $\\widehat{\\operatorname{Cov}}(z, y) = 0.6$ and $\\widehat{\\operatorname{Cov}}(z, x) = 0.3$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Expand the condition: $z^{\\top}y - z^{\\top}x\\,\\beta = 0$.\n", "\n", "**Step 2.** Solve: $\\hat\\beta = (z^{\\top}x)^{-1}z^{\\top}y$. Dividing numerator and denominator by $n - 1$ turns these into sample covariances.\n", "\n", "**Step 3.** $\\hat\\beta = 0.6/0.3 = 2$.\n", "\n", "**Answer:** 2, the Wald ratio from Day 12. With several instruments and regressors, $\\hat\\beta = (Z^{\\top}X)^{-1}Z^{\\top}y$, and a weak first stage ($Z^{\\top}X$ near singular) is an ill-conditioned system like Exercise 13.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "A = np.array([[3, 1], [2, 4.]])\n", "print(\"det =\", round(np.linalg.det(A), 6), \"\\ninverse =\\n\", np.linalg.inv(A))\n", "print(\"solution:\", np.linalg.solve(A, [9, 16]))\n", "print(\"3x3 det:\", round(np.linalg.det(np.array([[2, 0, 1], [1, 3, 2], [1, 1, 1.]])), 10))\n", "\n", "M = np.array([[1, 2], [1, -3.]])\n", "print(\"equilibrium (Q, P):\", np.linalg.solve(M, [120, 20]), \" d/dY:\", np.linalg.solve(M, [0.5, 0]))\n", "\n", "B = np.array([[1, 1], [1, 1.001]])\n", "print(\"ill-conditioned:\", np.linalg.solve(B, [2, 2.001]), np.linalg.solve(B, [2, 2.002]),\n", " \" condition number:\", round(np.linalg.cond(B)))\n", "\n", "L = np.array([[0.2, 0.3], [0.4, 0.1]])\n", "print(\"Leontief output:\", np.linalg.solve(np.eye(2) - L, [100, 50]).round(2))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- $Ax = b$ has a unique solution iff $\\det A \\ne 0$, iff $A$ has full rank.\n", "- $|\\det A|$ is an area/volume scale factor; a determinant near zero means a nearly flat (fragile) map.\n", "- Equilibrium models, input–output models and IV estimators are all linear systems." ] } ] }