Exponential Family and Link Functions

Generalized linear models extend ordinary regression by letting the response follow any distribution from one specific family and connecting its mean to predictors through a chosen link. Pick the distribution from the response type, then pick the link to keep predictions in the right range.

A response $Y$ belongs to the exponential dispersion family when its density or mass function factors into a specific shape. The general form has four pieces: data $y$, a natural parameter $\theta$, a dispersion $\varphi$, and helper functions $a$, $b$, and $c$.

$$f(y;\theta ,\varphi )=\exp \left\{\frac{y\theta -b(\theta )}{a(\varphi )}+c(y,\varphi )\right\}$$

Two derivatives of $b(\theta )$ carry all the structure you need:

$$E[Y]=b^{\prime }(\theta )=\mu ,\mathrm{Var}(Y)=b^{\prime \prime }(\theta )a(\varphi )$$

The piece $b^{\prime \prime }(\theta )$ written as a function of $\mu$ is called the variance function $V(\mu )$. It tells you how the variance scales with the mean, which is the fingerprint of each family.

KEY: Two derivatives of $b(\theta )$ give you the mean and variance. Memorize $b^{\prime }(\theta )=\mu$ and $V(\mu )=b^{\prime \prime }(\theta )$.

Common mistakes

• Logging the response instead of using a log link. Fitting OLS on $\log Y$ targets $E[\log Y]$, not $\log E[Y]$. The log link in a Poisson or gamma GLM targets $\log E[Y]$ directly. The two give different coefficients, and only the GLM handles $Y=0$ cleanly.
• Pairing Poisson with a binary response. Variance is $\mu (1-\mu )\le 0.25$, not $\mu$. Fitted means can exceed 1. Use binomial, not Poisson.
• Forgetting the exposure offset. Modeling raw claim counts without $\log E_i$ treats a 1-month policy the same as a 12-month policy. Always offset by log-exposure for frequency models.

Bottom line

• A distribution is in the exponential family if $f(y;\theta ,\varphi )=\exp \{[y\theta -b(\theta )]/a(\varphi )+c(y,\varphi )\}$. Members: normal, binomial, Poisson, gamma, inverse Gaussian, and negative binomial (fixed dispersion).
• $\theta$ is the canonical (natural) parameter; $\varphi$ the dispersion; $b^{\prime }(\theta )=\mu$; $b^{\prime \prime }(\theta )\cdot a(\varphi )=\mathrm{Var}(Y)$.
• The link $g$ connects linear predictor to mean: $g(\mu )=X\beta$. The canonical link sets $\theta =X\beta$, so the linear predictor equals the natural parameter.
• Canonical links by family: Normal = identity; Binomial = logit; Poisson = log; Gamma = inverse (reciprocal); Inverse Gaussian = inverse squared.

Exam shortcut

Match the response support to the family in one pass: real line → normal; $\{0,1\}$ or proportion → binomial; non-negative integers → Poisson (or negative binomial if overdispersed); strictly positive continuous → gamma or inverse Gaussian. Then pick the canonical link from the six-row table unless the prompt specifies otherwise.

The full lesson (about 2,783 words, 19 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• 2b

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.