Bagging, Boosting, and Random Forests

Free SOA Exam SRM (Statistics for Risk Modeling) lesson in Decision Trees. 21 min read, ~3,224 words.

Bagging fits B trees on bootstrap samples and averages (or majority-votes) them, reducing variance but not bias. Random forests add a per-split predictor subsample ( for classification, for regression) that decorrelates the trees. Boosting grows small trees sequentially on residuals with a shrinkage rate, and can overfit if B is...