Principal Components Analysis

You have 30 correlated risk factors and want a handful of synthetic axes that capture most of the variation. Principal component analysis (PCA) gives you those axes by eigendecomposing the predictor covariance matrix.

Given p predictors $X_1, \dots, X_p$, the first principal component $Z_1$ is the normalized linear combination $Z_1 = \phi_{11} X_1 + \phi_{21} X_2 + \dots + \phi_{p1} X_p$ with $\sum_j \phi_{j1}^2 = 1$ that has the largest sample variance. The coefficients $\phi_{j1}$ are the loadings of PC1; stacked, they form the loading vector $\phi_1$.

The second PC $Z_2$ is the highest-variance normalized linear combination uncorrelated with $Z_1$. Equivalently, $\phi_2$ is orthogonal to $\phi_1$. You can extract up to $\min(n-1, p)$ components in total.

KEY: PCs are new axes, not new variables in the original sense. They live in the same p-dimensional space but rotate it so the first axis points along maximum spread.

PCA is computed on centered data: subtract each predictor's mean so the cloud sits at the origin. Otherwise the first PC just points at the mean.

Common mistakes

• Skipping centering. PCA on uncentered data forces PC1 toward the mean vector, not the direction of maximum spread.
• Forgetting to standardize mixed-unit predictors. A predictor measured in dollars (variance in millions) will swamp one measured as a proportion (variance < 1) and capture a spurious 90%+ of variance.
• Confusing loadings with scores. Loadings are the coefficients $\phi_{jm}$ (one set per component); scores are the projected values $z_{im}$ (one per observation per component).

Bottom line

• Principal components are linear combinations of the original predictors that capture maximum variance, each uncorrelated with earlier components.
• The first PC is the direction of greatest data variation; the second PC is the highest-variance direction orthogonal to it.
• Loadings are the coefficients (unit-norm eigenvectors of the covariance or correlation matrix); scores are the projected values for each observation.
• Each eigenvalue equals the variance of its component score: $\mathrm{Var}(Z_m) = \lambda_m$.

Exam shortcut

If the question asks "how much variance does PC m explain?" compute $\lambda_m / \sum \lambda_j$ and stop. If predictors have mixed units, assume standardization and total variance = p. If a scree plot shows an obvious bend after component k, keep k components and treat the rest as noise.

The full lesson (about 2,913 words, 19 min read) adds 5 worked examples, all 7 common mistakes, a self-check, free in the app.

Learning objectives

• 5a

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.