Understand and calculate AIC, BIC, deviance, and R-squared.

Two GLMs fit the same loss data. One has a lower deviance, the other a lower AIC, and a colleague quotes a third number called BIC that disagrees with both. The exam expects you to know exactly what each statistic measures, how to compute it, and which one to trust for the question at hand.

Log-likelihood is the starting fuel. Every fit statistic below is built from $\ell =\ln L(\hat{\beta })$, the maximized log-likelihood. For Gaussian ordinary least squares (OLS) this is a function of SSE; for Poisson and binomial GLMs it has a clean closed form. You almost never compute $\ell$ from scratch on the exam. You take it as given and combine it with $k$ (parameter count, including the intercept and any dispersion parameter when stated) and $n$ (sample size).

Akaike Information Criterion (AIC). Trades fit against complexity.

$$\text{AIC}=-2\ell +2k$$

The 2k penalty comes from a Kullback-Leibler approximation: adding a parameter improves fit, but only an improvement of at least 1 unit of log-likelihood justifies it. Lower AIC wins.

Common mistakes

• Forgetting the dispersion parameter when counting $k$ for AIC. For Gaussian and gamma GLMs, ${\sigma }^2$ (or $\varphi$) is estimated and should be counted. Many exam keys treat it explicitly; read the prompt and add 1 if the variance is being estimated, not assumed.
• Using $\ln 10\approx 2.30$ when the sample size is 100. It is $\ln 100\approx 4.61$. The BIC penalty per parameter at $n=100$ is 4.61, not 2.30.
• Reporting "deviance $<n-k$, therefore fit is good". That heuristic only applies to Poisson or binomial with non-sparse data and a known dispersion of 1. Sparse cells or estimated dispersion break the rule.

Bottom line

• AIC $=-2\ell +2k$; BIC $=-2\ell +k\ln n$. Lower is better, and BIC penalizes parameters harder once $n>7$ (since $\ln 8\approx 2.08>2$).
• Scaled deviance $D^{\ast }=2({\ell }_{\text{sat}}-{\ell }_{\text{model}})$ measures lack of fit versus the saturated model. Unscaled deviance $D=\varphi D^{\ast }$ uses the dispersion $\varphi$.
• Deviance compares the fitted model to the saturated model, smaller is better; for a Gaussian model the deviance equals SSE.
• Nested GLMs: difference in scaled deviance $\sim {\chi }_q^2$ where $q$ is the parameter-count gap, the GLM analog of the partial $F$-test.

Exam shortcut

When a problem gives $\ell$ and asks for AIC and BIC together, compute $-2\ell$ once and add 2k and $k\ln n$ separately; the arithmetic almost always involves the same base. For Poisson and binomial GLM comparisons, jump straight to the deviance drop and compare against ${\chi }_q^2$; you rarely need to compute AIC unless the prompt asks.

The full lesson (about 2,319 words, 15 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• C7

Browse all free MAS-I lessons or jump into free MAS-I practice questions.