Principal Components Analysis

You have 30 correlated risk factors and want a handful of synthetic axes that capture most of the variation. Principal component analysis (PCA) gives you those axes by eigendecomposing the predictor covariance matrix.

Given p predictors $X_1,\dots ,X_p$, the first principal component $Z_1$ is the normalized linear combination $Z_1={\varphi }_{11}{X}_1+{\varphi }_{21}{X}_2+\cdots +{\varphi }_{p1}{X}_p$ with ${\sum }_j{\varphi }_{j1}^2=1$ that has the largest sample variance. The coefficients ${\varphi }_{j1}$ are the loadings of PC1; stacked, they form the loading vector ${\varphi }_1$.

The second PC $Z_2$ is the highest-variance normalized linear combination uncorrelated with $Z_1$. Equivalently, ${\varphi }_2$ is orthogonal to ${\varphi }_1$. You can extract up to $\min (n-1,p)$ components in total.

KEY: PCs are new axes, not new variables in the original sense. They live in the same p-dimensional space but rotate it so the first axis points along maximum spread.

PCA is computed on centered data: subtract each predictor's mean so the cloud sits at the origin. Otherwise the first PC just points at the mean.

Common mistakes

• Skipping centering. PCA on uncentered data forces PC1 toward the mean vector, not the direction of maximum spread.
• Forgetting to standardize mixed-unit predictors. A predictor measured in dollars (variance in millions) will swamp one measured as a proportion (variance < 1) and capture a spurious 90%+ of variance.
• Confusing loadings with scores. Loadings are the coefficients ${\varphi }_{jm}$ (one set per component); scores are the projected values $z_{im}$ (one per observation per component).

Bottom line

• Principal components are linear combinations of the original predictors that capture maximum variance, each uncorrelated with earlier components.
• The first PC is the direction of greatest data variation; the second PC is the highest-variance direction orthogonal to it.
• Loadings are the coefficients (unit-norm eigenvectors of the covariance or correlation matrix); scores are the projected values for each observation.
• Each eigenvalue equals the variance of its component score: $\mathrm{Var}(Z_m)={\lambda }_m$.

Exam shortcut

If the question asks "how much variance does PC m explain?" compute ${\lambda }_m/\sum {\lambda }_j$ and stop. If predictors have mixed units, assume standardization and total variance = p. If a scree plot shows an obvious bend after component k, keep k components and treat the rest as noise.

The full lesson (about 2,913 words, 19 min read) adds 5 worked examples, all 7 common mistakes, a self-check, free in the app.

Learning objectives

• 5a

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.