Model and Variable Selection

Model selection on SRM is a stack of triage questions: what distribution generated Y, what link connects the mean to the predictors, which predictors enter and how, and which of two candidate models wins on a defensible criterion. Every exam problem in this scope tests one rung of that ladder.

A generalized linear model (GLM) has three pieces: a response distribution from the exponential family, a linear predictor $\eta =X\beta$, and a link function $g$ with $g(\mu )=\eta$. The link maps the mean $\mu$ of Y onto the real line where $X\beta$ lives.

KEY: Pick the distribution from the shape and support of Y. Counts are not normal. Binary is not normal. Forcing OLS on a 0/1 response produces fitted probabilities outside [0, 1].

The canonical link equates the natural parameter of the exponential family with $X\beta$, which gives clean score equations and the simplest variance structure.

TRAP: A "linear model" with a log link is still linear in the parameters. The link is non-linear in $\mu$, not in $\beta$.

Common mistakes

• Using AIC to compare models fit on different responses. In the syllabus form AIC is built from the response's residual sum of squares, so its units are the units of $Y$ squared.
• Running an LRT on non-nested models. The chi-square distribution under the null requires nesting. For non-nested ordinary linear candidates, use AIC or BIC. A gamma-against-log-normal severity comparison is not ordinary linear, so neither criterion is in scope there; judge that pair on held-out prediction error.
• Dropping a main effect while keeping its interaction. Removing $X_1$ while keeping $X_1{X}_2$ forces the slope on $X_1$ at $X_2=0$ to be zero, an almost-never-defensible constraint.

Bottom line

• Choose the response distribution first (counts to Poisson, binary to binomial, positive skew to gamma), then the canonical link unless fit or interpretation says otherwise.
• t-tests target one coefficient; F-tests and likelihood ratio tests target nested-model comparisons. Use t for a single slope, F or LRT for blocks.
• A log link makes coefficients multiplicative: $e^{{\beta }_j}$ is the rate ratio per unit increase.
• SRM uses the ordinary-linear-model forms only: $\text{AIC}=\frac{1}{n}(\text{RSS}+2d{\hat{\sigma }}^2)$ and $\text{BIC}=\frac{1}{n}(\text{RSS}+\ln (n)d{\hat{\sigma }}^2)$. BIC penalizes complexity harder once n > 7 because ln(n) > 2.

Exam shortcut

If the response is a count, jump straight to Poisson with log link unless the prompt names overdispersion (then negative binomial). For binary Y, jump to logistic with logit link. For positive skewed Y, jump to gamma with log link. If the question gives two log-likelihoods and asks "is the extra block significant," compute $\Lambda =2\Delta \log L$ and compare to chi-square with df = parameters added; this is faster than recomputing AIC.

The full lesson (about 6,116 words, 41 min read) adds 6 worked examples, all 9 common mistakes, a self-check, free in the app.

Learning objectives

• 2e

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.