Evaluate models developed using an extended linear model approach.

A pricing actuary fits three competing GLMs to severity data: a gamma log-link, an inverse-Gaussian log-link, and a Tweedie. The coefficients look reasonable in all three. The exam question is not "which fits in-sample" but "which one would you defend at renewal." Evaluation is the whole job.

What "evaluate" means here. Extended linear models include GLMs, GAMs, GLMMs, and penalized variants (ridge, lasso, elastic net). Evaluation answers four questions: does the model fit the data, are the assumptions met, is it better than a simpler alternative, and will it generalize. Each question has its own toolkit.

Deviance: the generalized linear model (GLM) analog of residual sum of squares. For ordinary least squares (OLS) you minimize $\sum (y_i-{\hat{y}}_i{)}^2$. For a GLM you maximize the log-likelihood. The deviance is twice the log-likelihood gap between the saturated model (one parameter per observation, perfect fit) and the fitted model.

$$D=2[\ell (\mathbf{y};\mathbf{y})-\ell (\hat{\mathbf{\mu }};\mathbf{y})]$$

Smaller deviance is better. The scaled deviance $D^{\ast }=D/\varphi$ is approximately ${\chi }_{n-p}^2$ under the null that the model is correct, where $\varphi$ is the dispersion parameter (1 for Poisson and binomial...

Common mistakes

• Treating AIC differences smaller than 2 as meaningful. A 1-point AIC gap is noise. Require at least 2 points to call a winner, 10 points to call it decisive.
• Using Pearson ${\chi }^2/(n-p)=1.05$ as evidence of overdispersion. Values within roughly $[0.8,1.2]$ are consistent with no dispersion problem. The threshold for concern is closer to 1.5, not 1.05.
• Running an LRT on non-nested models. The chi-squared distribution for $2\Delta \ell$ requires nesting. For non-nested GLMs, use AIC or BIC, not LRT.

Bottom line

• Deviance $D=2[{\ell }_{\text{sat}}-{\ell }_{\text{model}}]$ measures lack-of-fit; scaled deviance $D/\varphi$ is approximately ${\chi }_{n-p}^2$ when the model is correctly specified.
• AIC $=-2\ell +2p$ and BIC $=-2\ell +p\ln n$, lower is better; AIC gaps below 2 are noise, above 10 are decisive, and BIC penalizes complexity harder once $n>7$.
• Likelihood ratio test for nested GLMs: $2({\ell }_{\text{full}}-{\ell }_{\text{reduced}})\sim {\chi }_{p_f-p_r}^2$; the ${\chi }^2$ result requires nesting, so use AIC or BIC otherwise.
• Pearson ${\chi }^2/(n-p)$ far from 1 signals dispersion; values in roughly $[0.8,1.2]$ are fine, but above 1.5 refit with quasi-Poisson, negative binomial, or added covariates.

Exam shortcut

Compute $\hat{\varphi }=X^2/(n-p)$ first whenever a problem gives both Pearson ${\chi }^2$ and degrees of freedom; the answer to "is this model adequate" usually hinges on whether that ratio is near 1. For LRT versus AIC choices, default to LRT only when the models are nested and the dispersion is known or fixed; otherwise reach for AIC.

The full lesson (about 2,366 words, 16 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• C3

Browse all free MAS-I lessons or jump into free MAS-I practice questions.