Use the delta method to estimate the variance of the maximum likelihood estimator of a function of the parameter(s).

An MLE for $\theta$ is rarely what you want to report. You want a survival probability, a quantile, an LER, or a premium. The delta method converts the variance of $\hat\theta$ into the variance of $g(\hat\theta)$ in one line of calculus.

MLE invariance. The functional invariance property says the MLE travels through any function. If $\hat\theta$ maximizes the likelihood for $\theta$, then $g(\hat\theta)$ is the MLE for $\tau = g(\theta)$. You do not re-maximize. You substitute. This is half of the delta-method package. It gives you the point estimate. The other half (variance) needs calculus.

The univariate delta method. Asymptotically the MLE satisfies $\sqrt{n}(\hat\theta - \theta) \to N(0,\, 1/I(\theta))$, where $I(\theta)$ is the Fisher information per observation. For a smooth function $g$, a first-order Taylor expansion around $\theta$ gives $g(\hat\theta) \approx g(\theta) + g'(\theta)(\hat\theta - \theta)$. Variance flows through linearly.

$$\text{Var}\!\left(g(\hat\theta)\right) \approx \left[g'(\theta)\right]^{2}\,\text{Var}(\hat\theta)$$

In practice you plug $\hat\theta$ into both $g'(\theta)$ and $\text{Var}(\hat\theta)$. The estimated information $I(\hat\theta)$ gives $\widehat{\text{Var}}(\hat\theta) = 1/[nI(\hat\theta)]$, or you read it off the inverse observed information directly.

Common mistakes

• Dropping the cross-term $2 g_{1}g_{2}\sigma_{12}$ in the multivariate formula. Costs you the correlation correction. With Example 2 numbers, the SE is wrong by a factor of 2.5.
• Forgetting to square $g'(\theta)$. The formula is $[g'(\theta)]^{2}\,\text{Var}(\hat\theta)$, not $g'(\theta)\,\text{Var}(\hat\theta)$. The latter can be negative.
• Leaving $\theta$ symbolic in the final variance. You must plug $\hat\theta$ into both $g'$ and the information.

Bottom line

• MLE invariance: if $\hat\theta$ is the MLE of $\theta$, then $g(\hat\theta)$ is the MLE of $g(\theta)$, with no re-optimization.
• Univariate delta: $\text{Var}(g(\hat\theta)) \approx [g'(\theta)]^{2}\,\text{Var}(\hat\theta)$, with both $g'$ and the information evaluated at $\hat\theta$.
• Multivariate delta: $\text{Var}(g(\hat{\boldsymbol\theta})) \approx \nabla g^{\top}\boldsymbol\Sigma\,\nabla g$, keeping off-diagonals, where $\boldsymbol\Sigma$ is the inverse Fisher information (inverse Hessian of the negative log-likelihood).
• SE and CI: standard error is the square root of the delta-method variance; the 95% Wald CI is $g(\hat\theta) \pm 1.96\,\text{SE}$.

Exam shortcut

Write $\hat\tau = g(\hat\theta)$ first, then $\widehat{\text{Var}}(\hat\tau) = \nabla g^{\top}\boldsymbol\Sigma\,\nabla g$. Two lines if scalar, three if vector. Do not derive the information from scratch when the problem hands you $\boldsymbol\Sigma$. For one-parameter MLEs, memorize $\widehat{\text{Var}}(\hat\theta) = \hat\theta^{2}/n$ for Exponential and $\hat p(1-\hat p)/n$ for Bernoulli. These cover most univariate delta problems. When two parameters are negatively correlated (Gamma, Pareto, Lognormal), expect the cross-term to reduce the SE of a product or sum of the MLEs.

The full lesson (about 3,090 words, 21 min read) adds 3 worked examples, all 9 common mistakes, a self-check, free in the app.

Learning objectives

• 4c

Browse all free Exam ASTAM lessons or jump into free Exam ASTAM practice questions.