Assessing Model Accuracy

Free SOA Exam SRM (Statistics for Risk Modeling) lesson in Basics of Statistical Learning. 14 min read, ~2,108 words.

Training error always falls as flexibility rises; test error is U-shaped. Minimize test error, not training error. Regression uses mean squared error (MSE); classification uses misclassification rate, ROC/AUC, or log-loss. The Bayes classifier is the theoretical lower bound on classification error. Class imbalance breaks raw accuracy; report AUC, F1, or...