Interpreting PCA Results

A PCA printout dumps loadings, eigenvalues, and variance proportions on you all at once. The exam tests whether you can read each piece and name what the component means.

For $p$ standardized predictors PCA returns $p$ principal components. Each component $Z_k$ is a linear combination of the original variables:

$$Z_k={\varphi }_{1k}{X}_1+{\varphi }_{2k}{X}_2+\cdots +{\varphi }_{pk}{X}_p$$

The loading vector ${\varphi }_k=({\varphi }_{1k},\dots ,{\varphi }_{pk})$ has unit length: ${\sum }_j{\varphi }_{jk}^2=1$. Loadings come from the eigenvectors of the correlation matrix (or covariance matrix if you skipped standardization).

A loading is the weight that variable $X_j$ carries inside PC $k$. Three things matter:

• Magnitude. $|{\varphi }_{jk}|$ close to the maximum possible value tells you $X_j$ drives that component. Values near zero say $X_j$ is irrelevant to it.
• Sign. Variables loading with the same sign move together along the component. Opposite signs mean the component contrasts them.
• Pattern across variables. If PC1 has roughly equal positive loadings on every variable, it represents an overall size or level.

Common mistakes

• Interpreting the sign of a single loading as economically meaningful. Sign is identifiable only relative to other loadings in the same PC. Only relative signs matter.
• Adding PVEs across different datasets or rotations. PVE is defined relative to the total variance of the input set. PVEs do not transfer.
• Using the Kaiser $\lambda >1$ rule on raw covariance PCA. The rule assumes each variable contributes variance 1, which is only true after standardization.

Bottom line

• Loadings are the weights (eigenvector entries) variables get inside each principal component. Large absolute loading = strong influence on that PC; sign tells direction.
• Each loading vector has unit length: ${\sum }_j{\varphi }_{jk}^2=1$.
• PVE for PC $k$ equals ${\lambda }_k/\sum {\lambda }_j$, where ${\lambda }_k$ is the eigenvalue. PVEs sum to 1 across all $p$ components and are reported in decreasing order.
• Cumulative PVE is how you pick the number of PCs to keep. Common rules: 80% to 90% cumulative, or the scree-plot elbow.

Exam shortcut

If the question lists eigenvalues, divide each by $p$ (standardized) or by $\sum {\lambda }_j$ (raw) to get PVE in one step. To name a PC, scan the loading column and circle entries with the largest absolute value; ignore loadings under about 0.3. If a printout shows one component carrying over 95% of variance on multi-unit data, suspect missing standardization before you trust the result.

The full lesson (about 1,646 words, 11 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• 5b

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.