One company out of fifteen can push a slope coefficient from significant to insignificant. Identifying which observation does that, and why, is the whole job of influence analysis.
Two kinds of observations can distort a fitted regression. A high-leverage point has an extreme value of an independent variable (extreme X). An outlier has an extreme value of the dependent variable, producing a large residual (extreme Y). Neither is automatically a problem. An observation is influential only if deleting it substantially changes the estimated coefficients or the goodness-of-fit statistics. In a simple regression a scatterplot suffices; multiple regression needs quantitative measures.
Leverage measures how far observation i's independent-variable values sit from those variables' means. It ranges from 0 to 1, and leverages sum to k + 1 across all n observations, where k is the number of independent variables.
For outliers, the preferred tool is the studentized deleted residual. Delete observation i, refit on the remaining n − 1 points, compare the actual with that model's prediction, then scale...
Common mistakes
- Treating any outlier as influential. A point far from the mean X that still sits on the regression line changes nothing. Influence requires a material change in coefficients on deletion.
- Using n − k − 1 degrees of freedom. Studentized residuals use n − k − 2. With n = 15 and k = 2 that is 11, not 12, so 2.201 rather than 2.179.
- Including n dummies for n categories. Four sectors need three dummies. The full set is collinear with the intercept and the regression fails.
Bottom line
- Leverage: extreme X; flag when h exceeds 3(k + 1)/n, and leverages sum to k + 1
- Studentized residual: extreme Y; outlier if absolute value exceeds 3, influential if it exceeds critical t with n − k − 2 degrees of freedom
- Influence means deletion materially changes coefficients or fit; outliers and high-leverage points are not automatically influential
- Dummies: n categories require n − 1 dummies, and the omitted category is the base
Exam shortcut
Compute the two thresholds before reading the answer choices: 3(k + 1)/n for leverage and the critical t at n − k − 2 for residuals. A stem that says "unusual value of the independent variable" points to leverage; "large residual" or "unusual Y" points to studentized residuals. The classic trap answer uses n − k − 1 degrees of freedom.
The full lesson (about 1,852 words, 12 min read) adds 2 worked examples, all 5 common mistakes, a self-check, free in the app.
Learning objectives
- extensions of regression
Browse all free CFA Level II lessons or jump into free CFA Level II practice questions.