Interpreting PCA Results

A PCA printout dumps loadings, eigenvalues, and variance proportions on you all at once. The exam tests whether you can read each piece and name what the component means.

For $p$ standardized predictors PCA returns $p$ principal components. Each component $Z_k$ is a linear combination of the original variables:

$$Z_k = \phi_{1k} X_1 + \phi_{2k} X_2 + \cdots + \phi_{pk} X_p$$

The loading vector $\phi_k = (\phi_{1k}, \dots, \phi_{pk})$ has unit length: $\sum_j \phi_{jk}^2 = 1$. Loadings come from the eigenvectors of the correlation matrix (or covariance matrix if you skipped standardization).

A loading is the weight that variable $X_j$ carries inside PC $k$. Three things matter:

• Magnitude. $|\phi_{jk}|$ close to the maximum possible value tells you $X_j$ drives that component. Values near zero say $X_j$ is irrelevant to it.
• Sign. Variables loading with the same sign move together along the component. Opposite signs mean the component contrasts them.
• Pattern across variables. If PC1 has roughly equal positive loadings on every variable, it represents an overall size or level.

Common mistakes

• Interpreting the sign of a single loading as economically meaningful. Sign is identifiable only relative to other loadings in the same PC. Only relative signs matter.
• Adding PVEs across different datasets or rotations. PVE is defined relative to the total variance of the input set. PVEs do not transfer.
• Using the Kaiser $\lambda > 1$ rule on raw covariance PCA. The rule assumes each variable contributes variance 1, which is only true after standardization.

Bottom line

• Loadings are the weights (eigenvector entries) variables get inside each principal component. Large absolute loading = strong influence on that PC; sign tells direction.
• Each loading vector has unit length: $\sum_j \phi_{jk}^2 = 1$.
• PVE for PC $k$ equals $\lambda_k / \sum \lambda_j$, where $\lambda_k$ is the eigenvalue. PVEs sum to 1 across all $p$ components and are reported in decreasing order.
• Cumulative PVE is how you pick the number of PCs to keep. Common rules: 80% to 90% cumulative, or the scree-plot elbow.

Exam shortcut

If the question lists eigenvalues, divide each by $p$ (standardized) or by $\sum \lambda_j$ (raw) to get PVE in one step. To name a PC, scan the loading column and circle entries with the largest absolute value; ignore loadings under about 0.3. If a printout shows one component carrying over 95% of variance on multi-unit data, suspect missing standardization before you trust the result.

The full lesson (about 1,646 words, 11 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• 5b

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.