Five correlated square-footage and unit-count variables clutter a home-price model. Principal components analysis collapses them into a handful of uncorrelated axes that keep most of the information, and the exam rewards the candidate who reads the output correctly.
Why PCA has value. When predictors are highly correlated, they carry redundant information and destabilize regression coefficients. PCA rotates the data onto new axes, the principal components, that are uncorrelated with each other. The first few often retain most of the variance, so you can drop the rest. That reduces dimensionality, tames multicollinearity, and can sharpen a downstream model (principal components regression). It is an unsupervised technique: it uses only the predictors, never the target.
KEY: PCA is a variance-preserving rotation. PC1 is the direction of maximum variance in the data cloud. PC2 is the direction of maximum remaining variance, perpendicular to PC1, and so on.
Loadings and computing a PC. Each component is a linear combination of the standardized variables. The coefficients are the loadings.
Common mistakes
- Forgetting to standardize. With square feet in the thousands and unit counts in single digits, unstandardized PCA yields a PC1 that just tracks square footage. Always standardize when units differ.
- Ignoring loading signs. Reading only magnitudes misses that opposite-sign loadings mean the component contrasts groups. A −0.49 next to +0.50 flips the interpretation from "size" to "contrast."
- Confusing a scree elbow with a cluster elbow. October 2025 candidates who read the PCA scree plot as a K-means cluster diagnostic earned no credit. The elbow selects components here, not clusters.
Bottom line
- PCA replaces correlated predictors with uncorrelated principal components ordered by variance explained.
- PC1 captures the most variance; each later PC captures the most of what remains, all mutually orthogonal.
- Standardize first (mean 0, variance 1) whenever variables use different units, or high-scale variables dominate.
- Loadings are the weights defining each PC; same-sign large loadings mean a shared underlying dimension.
Exam shortcut
Square each PC's standard deviation, divide by the total (which equals the number of variables for standardized data), and you have PVE instantly; then read the cumulative sum against your threshold. Read loading signs before magnitudes: all-same-sign means a "size" axis, mixed signs mean a "contrast" axis, and say which.
The full lesson (about 1,785 words, 12 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.
Learning objectives
- 3b
Browse all free Exam PA lessons or jump into free Exam PA practice questions.