The Bias-Variance Tradeoff

Free SOA Exam SRM (Statistics for Risk Modeling) lesson in Basics of Statistical Learning. 10 min read, ~1,445 words.

Expected test MSE decomposes into the variance of, the squared bias of, and irreducible error, all non-negative. More flexibility lowers bias and raises variance, and you cannot push both below a method-specific floor. Test MSE is U-shaped in flexibility, while training MSE falls monotonically and never marks the sweet spot...