You fit a straight line and it misses the curve; you fit a wiggly spline and it chases noise. Neither is best. The exam wants you to name why, using bias, variance, and complexity.
Bias. Bias is the error baked into a model by wrong or overly simple assumptions. A linear model forced onto a curved truth is biased no matter how much data you feed it. High bias means underfitting: the model is too rigid to capture the real relationship.
Variance. Variance measures sensitivity to the particular training set. Refit the same flexible model on a fresh sample and the predictions jump around. High variance means overfitting: the model memorizes noise, so it generalizes poorly.
Model complexity. Complexity, also called flexibility, is how many shapes the model can bend into. Adding polynomial terms, tree depth, or spline knots increases flexibility. More flexibility drives bias down and variance up. That tension is the whole trade-off.
KEY: Bias and variance move in opposite directions as complexity rises. You cannot minimize both at once; you minimize their sum.
Common mistakes
- Adding bias instead of bias squared. The decomposition uses . Using bias 0.7 directly instead of 0.7 squared corrupts the total.
- Calling high variance "high bias." A model that swings across refits has high variance, not bias. Bias is the systematic gap from truth.
- Thinking more flexibility is always better. Beyond the U-curve minimum, added complexity raises test error through variance.
Bottom line
- Bias is error from oversimplifying; a too rigid model underfits and misses the true signal.
- Variance is how much the fitted model swings when the training sample changes; flexible models overfit.
- Model complexity (flexibility) lowers bias but raises variance.
- Expected test MSE equals variance plus bias squared plus irreducible error Var(ε).
Exam shortcut
Remember the sum in three words: variance, bias squared, noise. When a prompt says "sensitive to the training set," answer variance; when it says "too simple to fit the pattern," answer bias. If test error rises as you add flexibility, variance is winning the trade-off, so step back toward the U-curve minimum rather than adding complexity.
The full lesson (about 1,185 words, 8 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.
Learning objectives
- 1c
Browse all free Exam PA lessons or jump into free Exam PA practice questions.