OLS vs GLM: Model Assumptions

An insurer fits a straight-line model to claim counts and gets negative predictions. The fix is not a better straight line; it is recognizing that OLS and GLMs rest on different assumption sets, and counts belong to the GLM world.

Ordinary least squares fits the model

$$Y_i={\beta }_0+{\beta }_1{X}_{i1}+\cdots +{\beta }_p{X}_{ip}+{\epsilon }_i,{\epsilon }_i\stackrel{iid}{\sim }N(0,{\sigma }^2)$$

That single equation packs five assumptions, traditionally labeled with the acronym LINE-M: Linearity of the mean, Independence of errors, Normality of errors, Equal variance (homoscedasticity), and no perfect Multicollinearity among predictors.

KEY: OLS does not assume the predictors are normal. It assumes the errors are normal. The X columns can be skewed, binary, or categorical.

Estimation in OLS. The coefficient vector minimizes the residual sum of squares,

$$\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$$

Under the LINE-M assumptions, $\hat{\beta }$ is the Best Linear Unbiased Estimator (Gauss-Markov) and, with normality added, the maximum likelihood estimator as well.

Common mistakes

• Claiming OLS requires normal predictors. OLS requires normal errors, not normal X. Categorical predictors are fine.
• Treating the log-link GLM as identical to OLS on log(Y). The first models $\log E[Y]$; the second models $E[\log Y]$. They give different coefficients and different fitted means.
• Forgetting independence still applies to GLMs. GLMs relax distribution and variance, not independence. Clustered claims need GEEs or mixed models even with a GLM.

Bottom line

• OLS assumes a normal, constant-variance error and a linear mean: $Y=X\beta +\epsilon$, $\epsilon \sim N(0,{\sigma }^2)$. Violate any of these and inference is unreliable.
• GLMs relax three OLS assumptions: the response can be any exponential-family distribution, variance can depend on the mean via $\text{Var}(Y)=\varphi V(\mu )$, and a link $g(\mu )=X\beta$ puts the linear predictor on a different scale than $\mu$.
• OLS estimates $\beta$ by minimizing residual sum of squares (closed form); GLMs estimate $\beta$ by maximum likelihood, usually via iteratively reweighted least squares (IRLS).
• OLS still requires independence of observations, correct mean structure, and no perfect multicollinearity. GLMs keep all three.

Exam shortcut

If the response is a count or a 0/1 indicator, OLS is wrong before you read the rest of the prompt; pick the matching GLM (Poisson or binomial). If a question asks "what does a GLM relax compared to OLS?" the three-word answer is distribution, variance, link.

The full lesson (about 4,521 words, 30 min read) adds 5 worked examples, all 9 common mistakes, a self-check, free in the app.

Learning objectives

• 2a

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.