Exponential Family and Link Functions

Generalized linear models extend ordinary regression by letting the response follow any distribution from one specific family and connecting its mean to predictors through a chosen link. Pick the distribution from the response type, then pick the link to keep predictions in the right range.

A response $Y$ belongs to the exponential dispersion family when its density or mass function factors into a specific shape. The general form has four pieces: data $y$, a natural parameter $\theta$, a dispersion $\phi$, and helper functions $a$, $b$, and $c$.

$$f(y; \theta, \phi) = \exp\!\left\{ \frac{y\theta - b(\theta)}{a(\phi)} + c(y, \phi) \right\}$$

Two derivatives of $b(\theta)$ carry all the structure you need:

$$E[Y] = b'(\theta) = \mu, \qquad \mathrm{Var}(Y) = b''(\theta)\, a(\phi)$$

The piece $b''(\theta)$ written as a function of $\mu$ is called the variance function $V(\mu)$. It tells you how the variance scales with the mean, which is the fingerprint of each family.

KEY: Two derivatives of $b(\theta)$ give you the mean and variance. Memorize $b'(\theta) = \mu$ and $V(\mu) = b''(\theta)$.

Common mistakes

• Logging the response instead of using a log link. Fitting OLS on $\log Y$ targets $E[\log Y]$, not $\log E[Y]$. The log link in a Poisson or gamma GLM targets $\log E[Y]$ directly. The two give different coefficients, and only the GLM handles $Y = 0$ cleanly.
• Pairing Poisson with a binary response. Variance is $\mu(1-\mu) \le 0.25$, not $\mu$. Fitted means can exceed 1. Use binomial, not Poisson.
• Forgetting the exposure offset. Modeling raw claim counts without $\log E_i$ treats a 1-month policy the same as a 12-month policy. Always offset by log-exposure for frequency models.

Bottom line

• A distribution is in the exponential family if $f(y;\theta,\phi) = \exp\{[y\theta - b(\theta)]/a(\phi) + c(y,\phi)\}$. Members: normal, binomial, Poisson, gamma, inverse Gaussian, and negative binomial (fixed dispersion).
• $\theta$ is the canonical (natural) parameter; $\phi$ the dispersion; $b'(\theta) = \mu$; $b''(\theta) \cdot a(\phi) = \mathrm{Var}(Y)$.
• The link $g$ connects linear predictor to mean: $g(\mu) = X\beta$. The canonical link sets $\theta = X\beta$, so the linear predictor equals the natural parameter.
• Canonical links by family: Normal = identity; Binomial = logit; Poisson = log; Gamma = inverse (reciprocal); Inverse Gaussian = inverse squared.

Exam shortcut

Match the response support to the family in one pass: real line → normal; $\{0,1\}$ or proportion → binomial; non-negative integers → Poisson (or negative binomial if overdispersed); strictly positive continuous → gamma or inverse Gaussian. Then pick the canonical link from the six-row table unless the prompt specifies otherwise.

The full lesson (about 2,783 words, 19 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• 2b

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.