Linear Regression with One and Multiple Regressors

You estimate a beta of 1.18 with a standard error of 0.31 over 60 months. The exam tests whether you can read that output, decide if beta is significantly different from one, and avoid the assumption traps that turn a clean regression into a misleading one.

Linear regression estimates models that are linear in the parameters. The functional form of the regressors can be non-linear:

• $Y={\beta }_0+{\beta }_{1}X+\epsilon$, standard linear
• $Y={\beta }_0+{\beta }_1\ln (X)+\epsilon$. Log regressor (still linear in $\beta$)
• $Y={\beta }_0+{\beta }_{1}X+{\beta }_2{X}^2+\epsilon$. Polynomial (still linear in $\beta$)

What it CANNOT estimate by ordinary least squares (OLS): $Y={\beta }_0\cdot e^{{\beta }_{1}X}+\epsilon$. Non-linear in ${\beta }_1$. Use non-linear least squares or transform first (take logs of both sides if appropriate).

Ordinary Least Squares chooses $\hat{\beta }$ to minimize the sum of squared residuals:

$$\underset{{\beta }_0,{\beta }_1}{\min }\sum _{i=1}^n{\left(Y_i-{\beta }_0-{\beta }_1{X}_i\right)}^2$$

Common mistakes

• Forgetting that adding regressors never lowers $R^2$. Use adjusted $R^2$ (or AIC/BIC) to compare models with different $k$. Trap: a question shows two models with $R^2=0.62$ and $R^2=0.65$ and asks which is better. Without adjusted $R^2$, the comparison is meaningless.
• Mistaking heteroskedasticity for bias. Heteroskedasticity does NOT bias coefficient estimates; it invalidates standard errors. Trap: "the regression has heteroskedasticity, so the estimated beta is biased". Wrong. The estimate is still unbiased, but the t-stat is misleading. Fix with White / robust standard errors.
• Calling individually-insignificant joint-significant regressors "useless." Multicollinearity: each individual t-stat near zero, joint F-test highly significant, high $R^2$. The variables together carry information; each alone looks redundant. Trap: dropping all of them tanks fit.

Bottom line

• OLS estimates models linear in the parameters; non-linear functions of $X$ (logs, polynomials) are fine, and the slope minimizes the sum of squared residuals.
• Six OLS assumptions: linearity, no multicollinearity, exogeneity ($E[\epsilon \mid X]=0$), homoskedasticity, no autocorrelation, normality. First five give BLUE (Gauss-Markov); normality enables exact small-sample t and F tests.
• Coefficient inference: $t=(\hat{\beta }-{\beta }_0)/\text{SE}(\hat{\beta })$; reject $H_0$ when $|t|$ exceeds the critical value (≈ 1.96 for large samples at 5%).
• Goodness of fit: $R^2$ = ESS/TSS = 1 - SSR/TSS. Adding regressors never lowers it, so only adjusted $R^2$ (or AIC/BIC) compares models with different $k$.

Exam shortcut

When a question shows a regression output and asks if a coefficient is "significantly different from one," recompute the t-statistic by hand: subtract 1 from the estimate before dividing by SE. The default printed t-stat tests against zero, not one. That is a frequent trap on CAPM beta questions.

The full lesson (about 2,829 words, 19 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15

Browse all free FRM Part I lessons or jump into free FRM Part I practice questions.