{ "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "colab": { "provenance": [] } }, "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Day 23: Derivatives, Partial Derivatives & Taylor Series\n", "\n", "*Beginner to Ninja: Math for Causal Inference*\n", "\n", "Rates of change, marginal effects and elasticities, and local polynomial approximations of any smooth function.\n", "\n", "**Learning outcomes**\n", "\n", "- Differentiate with the product, quotient and chain rules and compute partial derivatives.\n", "- Compute marginal effects and elasticities in linear, log and non-linear models.\n", "- Build first- and second-order Taylor approximations in one and several variables.\n", "- Explain the log-point approximation and when it breaks down." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Theory\n", "\n", "### The derivative\n", "\n", "$$\n", "f'(x) = \\lim_{h \\to 0}\\frac{f(x + h) - f(x)}{h}\n", "$$\n", "\n", "is the instantaneous rate of change: the slope of the tangent line. Rules you need:\n", "\n", "| Rule | Formula |\n", "|---|---|\n", "| Power | $(x^n)' = nx^{n-1}$ |\n", "| Exponential, log | $(e^{x})' = e^{x}$, $(\\ln x)' = 1/x$ |\n", "| Product | $(fg)' = f'g + fg'$ |\n", "| Quotient | $(f/g)' = (f'g - fg')/g^2$ |\n", "| Chain | $(f(g(x)))' = f'(g(x))\\,g'(x)$ |\n", "\n", "### Partial derivatives and the gradient\n", "\n", "For $f(x_1, \\dots, x_k)$, the **partial derivative** $\\partial f/\\partial x_j$ holds every other variable fixed: *ceteris paribus* in symbols. The **gradient** $\\nabla f$ stacks the partials; the **Hessian** $H$ collects the second partials $\\partial^2 f/\\partial x_j\\partial x_\\ell$ (symmetric for smooth $f$).\n", "\n", "### Marginal effects and elasticities\n", "\n", "- Marginal effect: $\\partial E[y \\mid x]/\\partial x_j$. In a linear model it is $\\beta_j$; with interactions or non-linear links it depends on $x$.\n", "- Elasticity: $\\varepsilon = \\frac{\\partial y}{\\partial x}\\frac{x}{y} = \\frac{\\partial\\ln y}{\\partial\\ln x}$.\n", "- Semi-elasticity: $\\partial\\ln y/\\partial x$, the coefficient in a log-level regression.\n", "\n", "### Taylor series\n", "\n", "Near a point $a$,\n", "\n", "$$\n", "f(x) \\approx f(a) + f'(a)(x - a) + \\tfrac12 f''(a)(x - a)^2,\n", "$$\n", "\n", "and in several variables\n", "\n", "$$\n", "f(x) \\approx f(a) + \\nabla f(a)^{\\top}(x - a) + \\tfrac12(x - a)^{\\top}H(a)(x - a).\n", "$$\n", "\n", "The first-order term is the delta method (Day 12); the second-order term is the curvature that Newton's method (Day 25) and Fisher information (Day 11) use. The famous **log-point approximation** is the first-order expansion $\\ln(1 + x) \\approx x$, good when $x$ is small.\n", "\n", "## Part 1 · Warm-up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 1 · A polynomial\n", "\n", "Differentiate $f(x) = 3x^2 + 2x$.\n", "\n", "
Show solution\n", "\n", "$f'(x) = 6x + 2$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 2 · Chain rule\n", "\n", "Differentiate $e^{2x}$ and $\\ln(x^2 + 1)$.\n", "\n", "
Show solution\n", "\n", "$2e^{2x}$ and $2x/(x^2 + 1)$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 3 · Partial derivatives\n", "\n", "For $f(x, y) = x^2y + 3y^2$, find $f_x$, $f_y$ and the Hessian.\n", "\n", "
Show solution\n", "\n", "$f_x = 2xy$, $f_y = x^2 + 6y$. $H = \\begin{pmatrix}2y & 2x\\\\2x & 6\\end{pmatrix}$ (note $f_{xy} = f_{yx} = 2x$).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 4 · A constant elasticity\n", "\n", "Demand is $Q = AP^{-1.5}$. Find the price elasticity.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Take logs: $\\ln Q = \\ln A - 1.5\\ln P$.\n", "\n", "**Step 2.** Elasticity $= \\partial\\ln Q/\\partial\\ln P = -1.5$.\n", "\n", "**Answer:** −1.5 at every price: a 1% price rise cuts quantity demanded by 1.5%.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 5 · A logit marginal effect\n", "\n", "In a logit, $P(y = 1 \\mid x) = \\Lambda(\\beta_0 + \\beta_1x)$ with $\\Lambda'(z) = \\Lambda(z)(1 - \\Lambda(z))$. If $\\beta_1 = 0.8$ and the predicted probability is 0.3, what is the marginal effect of $x$?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Chain rule: $\\partial P/\\partial x = \\Lambda'(\\beta_0 + \\beta_1x)\\,\\beta_1 = p(1 - p)\\beta_1$.\n", "\n", "**Step 2.** At $p = 0.3$: $0.3 \\times 0.7 \\times 0.8 = 0.168$.\n", "\n", "**Answer:** about 17 percentage points per unit of $x$ *at this point*; at $p = 0.5$ it would be $0.25 \\times 0.8 = 0.2$.\n", "\n", "
\n", "\n", "## Part 2 · Proofs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 6 · The product rule\n", "\n", "Prove $(fg)' = f'g + fg'$ from the definition.\n", "\n", "
Show solution\n", "\n", "$f(x + h)g(x + h) - f(x)g(x) = [f(x + h) - f(x)]g(x + h) + f(x)[g(x + h) - g(x)]$. Divide by $h$ and let $h \\to 0$; continuity of $g$ gives $g(x + h) \\to g(x)$, so the limit is $f'g + fg'$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 7 · The derivative of the log\n", "\n", "Using $e^{\\ln x} = x$ and the chain rule, prove $(\\ln x)' = 1/x$.\n", "\n", "
Show solution\n", "\n", "Differentiate both sides: $e^{\\ln x}(\\ln x)' = 1$, so $(\\ln x)' = 1/e^{\\ln x} = 1/x$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 8 · The error of a linear approximation\n", "\n", "Show that if $f''$ is bounded by $K$ near $a$, the first-order approximation error is at most $\\frac K2(x - a)^2$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Taylor's theorem with remainder: $f(x) = f(a) + f'(a)(x - a) + \\tfrac12 f''(c)(x - a)^2$ for some $c$ between $a$ and $x$.\n", "\n", "**Step 2.** So the error is $\\tfrac12|f''(c)|(x - a)^2 \\le \\tfrac K2(x - a)^2$.\n", "\n", "**Step 3.** It shrinks *quadratically*: halve the distance, quarter the error.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 9 · Cobb–Douglas and Euler's theorem\n", "\n", "For $Y = AK^{\\alpha}L^{1-\\alpha}$, show $\\partial\\ln Y/\\partial\\ln K = \\alpha$ and $KY_K + LY_L = Y$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\ln Y = \\ln A + \\alpha\\ln K + (1 - \\alpha)\\ln L$, so the elasticity is $\\alpha$.\n", "\n", "**Step 2.** Then $Y_K = \\alpha Y/K$ and $Y_L = (1 - \\alpha)Y/L$, so $KY_K + LY_L = \\alpha Y + (1 - \\alpha)Y = Y$: with constant returns, paying factors their marginal products exhausts output.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 10 · Second-order Taylor of the log\n", "\n", "Derive the second-order Taylor expansion of $\\ln(1 + x)$ at 0 and evaluate it at $x = 0.1$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Derivatives of $g(x) = \\ln(1 + x)$: $g' = 1/(1 + x)$, $g'' = -1/(1 + x)^2$.\n", "\n", "**Step 2.** At 0: $g(0) = 0$, $g'(0) = 1$, $g''(0) = -1$.\n", "\n", "**Step 3.** Second-order expansion: $\\ln(1 + x) \\approx 0 + x - \\tfrac12x^2$.\n", "\n", "**Step 4.** At $x = 0.1$: $0.1 - 0.005 = 0.095$, versus the true $\\ln 1.1 \\approx 0.09531$.\n", "\n", "**Answer:** $x - x^2/2$; error 0.0003 at $x = 0.1$, against 0.005 for the first-order $x$.\n", "\n", "
\n", "\n", "## Part 3 · Applied" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 11 · Log points versus percent\n", "\n", "A log-wage regression gives a training coefficient of 0.25. What percentage effect is that, and how wrong is the \"25%\" reading?\n", "\n", "
Show solution\n", "\n", "**Step 1.** A log-point coefficient $b$ means wages are multiplied by $e^{b}$.\n", "\n", "**Step 2.** $100(e^{0.25} - 1) = 100(1.284 - 1) \\approx 28.4\\%$.\n", "\n", "**Answer:** 28.4%, not 25%. The approximation $\\ln(1 + x) \\approx x$ understates large positive effects; below about 0.1 the gap is under half a point.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 12 · Returns to experience\n", "\n", "Wages follow $w = 10 + 2\\,\\text{exp} - 0.05\\,\\text{exp}^2$. Find the marginal effect of experience at 10 years and where it peaks.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $\\partial w/\\partial\\,\\text{exp} = 2 - 2(0.05)\\,\\text{exp} = 2 - 0.1\\,\\text{exp}$.\n", "\n", "**Step 2.** At 10 years: $2 - 1 = 1$ per extra year.\n", "\n", "**Step 3.** Peak: $2 - 0.1\\,\\text{exp} = 0 \\Rightarrow \\text{exp} = 20$ years (second derivative $-0.1 < 0$).\n", "\n", "**Answer:** the return is 1 per year at 10 years of experience, and the wage profile peaks at 20 years.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 13 · Heterogeneous treatment effects via interactions\n", "\n", "$y = \\beta_0 + \\beta_1T + \\beta_2X + \\beta_3TX$ with $\\beta_1 = 2$, $\\beta_3 = 0.5$. What is the effect of $T$ for a unit with $X = 4$? With $X = 0$?\n", "\n", "
Show solution\n", "\n", "**Step 1.** Differentiate with respect to $T$: $\\partial y/\\partial T = \\beta_1 + \\beta_3X$.\n", "\n", "**Step 2.** At $X = 4$: $2 + 0.5(4) = 4$. At $X = 0$: $2$.\n", "\n", "**Answer:** 4 and 2. The \"main effect\" $\\beta_1$ is the effect only where $X = 0$, which is why covariates are often centred before interacting.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 14 · Approximating the wage bill\n", "\n", "A region's wage bill (in \\$ billion) is $f(g, L) = e^{g}L$, where $g$ is the log growth of the average wage and $L$ is employment (millions). Start from $(g, L) = (0, 1)$. Approximate $f$ to first order and evaluate it at 10% wage growth ($g = 0.1$) and employment of 1.2 million; compare with the exact value.\n", "\n", "
Show solution\n", "\n", "**Step 1.** Value at the base point: $f(0, 1) = e^0 \\times 1 = 1$.\n", "\n", "**Step 2.** Partial derivatives: $f_g = e^{g}L = 1$ and $f_L = e^{g} = 1$ at $(0, 1)$.\n", "\n", "**Step 3.** First-order approximation: $f \\approx 1 + 1(g - 0) + 1(L - 1)$. At $(0.1, 1.2)$: $1 + 0.1 + 0.2 = 1.3$.\n", "\n", "**Step 4.** Exact: $e^{0.1} \\times 1.2 \\approx 1.105 \\times 1.2 = 1.326$.\n", "\n", "**Answer:** \\$1.30 billion approximately versus \\$1.326 billion exactly. The gap is the second-order (interaction) term: wage growth applies to the extra workers too.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise 15 · Back to the delta method\n", "\n", "Use a first-order Taylor expansion of $g(p) = p/(1 - p)$ around $p = 0.6$ to recover Day 12's standard error for the odds when $\\operatorname{SE}(\\hat p) = 0.0894$.\n", "\n", "
Show solution\n", "\n", "**Step 1.** $g(p) = p/(1 - p)$ has $g'(p) = 1/(1 - p)^2 = 1/0.16 = 6.25$ at $p = 0.6$.\n", "\n", "**Step 2.** First-order Taylor: $g(\\hat p) - g(p) \\approx 6.25(\\hat p - p)$.\n", "\n", "**Step 3.** So $\\operatorname{SE}(g(\\hat p)) \\approx 6.25 \\times 0.0894 \\approx 0.559$.\n", "\n", "**Answer:** 0.559, Day 12's result: the delta method is Taylor's theorem plus the CLT.\n", "\n", "
\n", "\n", "## Verify in Python" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Your work:*" ] }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import numpy as np\n", "\n", "# Numerical derivatives (central differences) versus the formulas\n", "d = lambda f, x, h=1e-6: (f(x + h) - f(x - h)) / (2 * h)\n", "print(\"d/dx ln(x^2+1) at 1:\", round(d(lambda x: np.log(x**2 + 1), 1.0), 6), \"formula:\", 2 * 1 / 2)\n", "\n", "lam = lambda z: 1 / (1 + np.exp(-z))\n", "z0 = np.log(0.3 / 0.7) # where the predicted probability is 0.3\n", "print(\"logit marginal effect:\", round(0.8 * d(lam, z0), 4))\n", "\n", "for x in (0.01, 0.1, 0.25, 0.5):\n", " print(f\"x={x}: ln(1+x)={np.log1p(x):.5f}, 1st order={x:.5f}, 2nd order={x - x**2/2:.5f}\")\n", "print(\"exact % effect of 0.25 log points:\", round(100 * (np.exp(0.25) - 1), 2))\n", "print(\"e^x y at (0.1, 1.2):\", round(np.exp(0.1) * 1.2, 4), \" first-order approx: 1.3\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Key takeaways\n", "\n", "- Derivatives give marginal effects; logs turn them into elasticities and semi-elasticities.\n", "- Taylor expansions replace any smooth function with a local line or parabola; the error shrinks quadratically.\n", "- The delta method, Newton's method and Fisher information are all Taylor expansions." ] } ] }