Day 23: Derivatives, Partial Derivatives & Taylor Series

Theory

The derivative

f'(x) = \lim_{h \to 0}\frac{f(x + h) - f(x)}{h}

is the instantaneous rate of change: the slope of the tangent line. Rules you need:

Rule	Formula
Power	$(x^n)' = nx^{n-1}$
Exponential, log	$(e^{x})' = e^{x}$ , $(\ln x)' = 1/x$
Product	$(fg)' = f'g + fg'$
Quotient	$(f/g)' = (f'g - fg')/g^2$
Chain	$(f(g(x)))' = f'(g(x))\,g'(x)$

Partial derivatives and the gradient

For $f(x_1, \dots, x_k)$ , the partial derivative $\partial f/\partial x_j$ holds every other variable fixed: ceteris paribus in symbols. The gradient $\nabla f$ stacks the partials; the Hessian $H$ collects the second partials $\partial^2 f/\partial x_j\partial x_\ell$ (symmetric for smooth $f$ ).

Marginal effects and elasticities

Marginal effect: $\partial E[y \mid x]/\partial x_j$ . In a linear model it is $\beta_j$ ; with interactions or non-linear links it depends on $x$ .
Elasticity: $\varepsilon = \frac{\partial y}{\partial x}\frac{x}{y} = \frac{\partial\ln y}{\partial\ln x}$ .
Semi-elasticity: $\partial\ln y/\partial x$ , the coefficient in a log-level regression.

Taylor series

Near a point $a$ ,

f(x) \approx f(a) + f'(a)(x - a) + \tfrac12 f''(a)(x - a)^2,

and in several variables

f(x) \approx f(a) + \nabla f(a)^{\top}(x - a) + \tfrac12(x - a)^{\top}H(a)(x - a).

The first-order term is the delta method (Day 12); the second-order term is the curvature that Newton's method (Day 25) and Fisher information (Day 11) use. The famous log-point approximation is the first-order expansion $\ln(1 + x) \approx x$ , good when $x$ is small.

Part 1 · Warm-up

Exercise 1 · A polynomial

Differentiate $f(x) = 3x^2 + 2x$ .

Show solution

$f'(x) = 6x + 2$ .

Exercise 2 · Chain rule

Differentiate $e^{2x}$ and $\ln(x^2 + 1)$ .

Show solution

$2e^{2x}$ and $2x/(x^2 + 1)$ .

Exercise 3 · Partial derivatives

For $f(x, y) = x^2y + 3y^2$ , find $f_x$ , $f_y$ and the Hessian.

Show solution

$f_x = 2xy$ , $f_y = x^2 + 6y$ . $H = \begin{pmatrix}2y & 2x\\2x & 6\end{pmatrix}$ (note $f_{xy} = f_{yx} = 2x$ ).

Exercise 4 · A constant elasticity

Demand is $Q = AP^{-1.5}$ . Find the price elasticity.

Show solution

Step 1. Take logs: $\ln Q = \ln A - 1.5\ln P$ .

Step 2. Elasticity $= \partial\ln Q/\partial\ln P = -1.5$ .

Answer: −1.5 at every price: a 1% price rise cuts quantity demanded by 1.5%.

Exercise 5 · A logit marginal effect

In a logit, $P(y = 1 \mid x) = \Lambda(\beta_0 + \beta_1x)$ with $\Lambda'(z) = \Lambda(z)(1 - \Lambda(z))$ . If $\beta_1 = 0.8$ and the predicted probability is 0.3, what is the marginal effect of $x$ ?

Show solution

Step 1. Chain rule: $\partial P/\partial x = \Lambda'(\beta_0 + \beta_1x)\,\beta_1 = p(1 - p)\beta_1$ .

Step 2. At $p = 0.3$ : $0.3 \times 0.7 \times 0.8 = 0.168$ .

Answer: about 17 percentage points per unit of $x$ at this point; at $p = 0.5$ it would be $0.25 \times 0.8 = 0.2$ .

Part 2 · Proofs

Exercise 6 · The product rule

Prove $(fg)' = f'g + fg'$ from the definition.

Show solution

$f(x + h)g(x + h) - f(x)g(x) = [f(x + h) - f(x)]g(x + h) + f(x)[g(x + h) - g(x)]$ . Divide by $h$ and let $h \to 0$ ; continuity of $g$ gives $g(x + h) \to g(x)$ , so the limit is $f'g + fg'$ .

Exercise 7 · The derivative of the log

Using $e^{\ln x} = x$ and the chain rule, prove $(\ln x)' = 1/x$ .

Show solution

Differentiate both sides: $e^{\ln x}(\ln x)' = 1$ , so $(\ln x)' = 1/e^{\ln x} = 1/x$ .

Exercise 8 · The error of a linear approximation

Show that if $f''$ is bounded by $K$ near $a$ , the first-order approximation error is at most $\frac K2(x - a)^2$ .

Show solution

Step 1. Taylor's theorem with remainder: $f(x) = f(a) + f'(a)(x - a) + \tfrac12 f''(c)(x - a)^2$ for some $c$ between $a$ and $x$ .

Step 2. So the error is $\tfrac12|f''(c)|(x - a)^2 \le \tfrac K2(x - a)^2$ .

Step 3. It shrinks quadratically: halve the distance, quarter the error.

Exercise 9 · Cobb–Douglas and Euler's theorem

For $Y = AK^{\alpha}L^{1-\alpha}$ , show $\partial\ln Y/\partial\ln K = \alpha$ and $KY_K + LY_L = Y$ .

Show solution

Step 1. $\ln Y = \ln A + \alpha\ln K + (1 - \alpha)\ln L$ , so the elasticity is $\alpha$ .

Step 2. Then $Y_K = \alpha Y/K$ and $Y_L = (1 - \alpha)Y/L$ , so $KY_K + LY_L = \alpha Y + (1 - \alpha)Y = Y$ : with constant returns, paying factors their marginal products exhausts output.

Exercise 10 · Second-order Taylor of the log

Derive the second-order Taylor expansion of $\ln(1 + x)$ at 0 and evaluate it at $x = 0.1$ .

Show solution

Step 1. Derivatives of $g(x) = \ln(1 + x)$ : $g' = 1/(1 + x)$ , $g'' = -1/(1 + x)^2$ .

Step 2. At 0: $g(0) = 0$ , $g'(0) = 1$ , $g''(0) = -1$ .

Step 3. Second-order expansion: $\ln(1 + x) \approx 0 + x - \tfrac12x^2$ .

Step 4. At $x = 0.1$ : $0.1 - 0.005 = 0.095$ , versus the true $\ln 1.1 \approx 0.09531$ .

Answer: $x - x^2/2$ ; error 0.0003 at $x = 0.1$ , against 0.005 for the first-order $x$ .

Part 3 · Applied

Exercise 11 · Log points versus percent

A log-wage regression gives a training coefficient of 0.25. What percentage effect is that, and how wrong is the "25%" reading?

Show solution

Step 1. A log-point coefficient $b$ means wages are multiplied by $e^{b}$ .

Step 2. $100(e^{0.25} - 1) = 100(1.284 - 1) \approx 28.4\%$ .

Answer: 28.4%, not 25%. The approximation $\ln(1 + x) \approx x$ understates large positive effects; below about 0.1 the gap is under half a point.

Exercise 12 · Returns to experience

Wages follow $w = 10 + 2\,\text{exp} - 0.05\,\text{exp}^2$ . Find the marginal effect of experience at 10 years and where it peaks.

Show solution

Step 1. $\partial w/\partial\,\text{exp} = 2 - 2(0.05)\,\text{exp} = 2 - 0.1\,\text{exp}$ .

Step 2. At 10 years: $2 - 1 = 1$ per extra year.

Step 3. Peak: $2 - 0.1\,\text{exp} = 0 \Rightarrow \text{exp} = 20$ years (second derivative $-0.1 < 0$ ).

Answer: the return is 1 per year at 10 years of experience, and the wage profile peaks at 20 years.

Exercise 13 · Heterogeneous treatment effects via interactions

$y = \beta_0 + \beta_1T + \beta_2X + \beta_3TX$ with $\beta_1 = 2$ , $\beta_3 = 0.5$ . What is the effect of $T$ for a unit with $X = 4$ ? With $X = 0$ ?

Show solution

Step 1. Differentiate with respect to $T$ : $\partial y/\partial T = \beta_1 + \beta_3X$ .

Step 2. At $X = 4$ : $2 + 0.5(4) = 4$ . At $X = 0$ : $2$ .

Answer: 4 and 2. The "main effect" $\beta_1$ is the effect only where $X = 0$ , which is why covariates are often centred before interacting.

Exercise 14 · Approximating the wage bill

A region's wage bill (in $ billion) is $f(g, L) = e^{g}L$ , where $g$ is the log growth of the average wage and $L$ is employment (millions). Start from $(g, L) = (0, 1)$ . Approximate $f$ to first order and evaluate it at 10% wage growth ( $g = 0.1$ ) and employment of 1.2 million; compare with the exact value.

Show solution

Step 1. Value at the base point: $f(0, 1) = e^0 \times 1 = 1$ .

Step 2. Partial derivatives: $f_g = e^{g}L = 1$ and $f_L = e^{g} = 1$ at $(0, 1)$ .

Step 3. First-order approximation: $f \approx 1 + 1(g - 0) + 1(L - 1)$ . At $(0.1, 1.2)$ : $1 + 0.1 + 0.2 = 1.3$ .

Step 4. Exact: $e^{0.1} \times 1.2 \approx 1.105 \times 1.2 = 1.326$ .

Answer: $1.30 billion approximately versus $1.326 billion exactly. The gap is the second-order (interaction) term: wage growth applies to the extra workers too.

Exercise 15 · Back to the delta method

Use a first-order Taylor expansion of $g(p) = p/(1 - p)$ around $p = 0.6$ to recover Day 12's standard error for the odds when $\operatorname{SE}(\hat p) = 0.0894$ .

Show solution

Step 1. $g(p) = p/(1 - p)$ has $g'(p) = 1/(1 - p)^2 = 1/0.16 = 6.25$ at $p = 0.6$ .

Step 2. First-order Taylor: $g(\hat p) - g(p) \approx 6.25(\hat p - p)$ .

Step 3. So $\operatorname{SE}(g(\hat p)) \approx 6.25 \times 0.0894 \approx 0.559$ .

Answer: 0.559, Day 12's result: the delta method is Taylor's theorem plus the CLT.

Verify in Python

import numpy as np

# Numerical derivatives (central differences) versus the formulas
d = lambda f, x, h=1e-6: (f(x + h) - f(x - h)) / (2 * h)
print("d/dx ln(x^2+1) at 1:", round(d(lambda x: np.log(x**2 + 1), 1.0), 6), "formula:", 2 * 1 / 2)

lam = lambda z: 1 / (1 + np.exp(-z))
z0 = np.log(0.3 / 0.7)                              # where the predicted probability is 0.3
print("logit marginal effect:", round(0.8 * d(lam, z0), 4))

for x in (0.01, 0.1, 0.25, 0.5):
    print(f"x={x}: ln(1+x)={np.log1p(x):.5f}, 1st order={x:.5f}, 2nd order={x - x**2/2:.5f}")
print("exact % effect of 0.25 log points:", round(100 * (np.exp(0.25) - 1), 2))
print("e^x y at (0.1, 1.2):", round(np.exp(0.1) * 1.2, 4), " first-order approx: 1.3")

Key takeaways

Derivatives give marginal effects; logs turn them into elasticities and semi-elasticities.
Taylor expansions replace any smooth function with a local line or parabola; the error shrinks quadratically.
The delta method, Newton's method and Fisher information are all Taylor expansions.

Day 23. Derivatives, Partial Derivatives & Taylor Series

Theory

The derivative

Partial derivatives and the gradient

Marginal effects and elasticities

Taylor series

Part 1 · Warm-up

Exercise 1 · A polynomial

Exercise 2 · Chain rule

Exercise 3 · Partial derivatives

Exercise 4 · A constant elasticity

Exercise 5 · A logit marginal effect

Part 2 · Proofs

Exercise 6 · The product rule

Exercise 7 · The derivative of the log

Exercise 8 · The error of a linear approximation

Exercise 9 · Cobb–Douglas and Euler's theorem

Exercise 10 · Second-order Taylor of the log

Part 3 · Applied

Exercise 11 · Log points versus percent

Exercise 12 · Returns to experience

Exercise 13 · Heterogeneous treatment effects via interactions

Exercise 14 · Approximating the wage bill

Exercise 15 · Back to the delta method

Verify in Python

Key takeaways

Simulation Lab: Taylor Approximations