How GLM Approaches Differ from OLS

Free SOA Exam SRM (Statistics for Risk Modeling) lesson in Linear Models. 24 min read, ~3,552 words.

OLS minimizes RSS unpenalized; ridge adds an penalty, lasso an penalty, and KNN abandons a functional form. Ridge shrinks coefficients toward zero but never to zero, while lasso shrinks and sets some exactly to zero, performing variable selection. Always standardize predictors before ridge or lasso, since their penalties are scale-sensitive...