Sample Moments and Hypothesis Testing

You compute the historical Sharpe ratio of a strategy at 1.2 over 24 months. The next desk over runs the same strategy and gets 0.6 over 36 months. The exam tests whether the gap is real or whether you'd expect that much variation by chance, and how to make that call without overstating your confidence.

A population moment is a fixed property of the true (unknown) distribution: the true mean $\mu$, the true variance $\sigma^2$, the true skewness, and so on. You don't observe these directly. A sample moment is computed from data:

$$\bar{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$$

$$s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \bar{X})^2$$

The $n - 1$ in the variance denominator (Bessel's correction) makes the sample variance unbiased. With $n$, you would systematically underestimate true variance, because you lost one degree of freedom estimating $\bar{X}$ from the same data.

The sample standard deviation is $s=\sqrt{s^2}$. Variance has squared units; standard deviation has the original units. Although $s^2$ is unbiased under independent, identically distributed sampling with finite variance, taking its square root does not...

Common mistakes

• Confusing Type I and Type II errors. Type I = reject true $H_0$. Type II = fail to reject false $H_0$. Trap: "what is the size of the test?" and a choice gives $\beta$. Size = $\alpha$ = Type I rate; power = $1 - \beta$.
• Interpreting p-value as $P(H_0 \text{ true})$. A p of 0.03 means data would be this extreme 3% of the time IF $H_0$ is true. Trap: "3% probability the strategy has zero alpha" conflates conditional with posterior.
• Treating one- and two-sided confidence intervals as interchangeable. A two-sided 5% test uses a 95% two-sided interval; a one-sided 5% test uses a 95% bound. The bound shares one endpoint with a two-sided 90% interval. Excluding the null on the opposite side is not evidence for the prespecified one-sided alternative.

Bottom line

• Population moments are fixed parameters; sample moments are random estimates. The sample mean is unbiased, consistent, and BLUE (Best Linear Unbiased Estimator), and the $n - 1$ divisor makes sample variance unbiased.
• Under independent, identically distributed sampling, LLN gives consistency with a finite mean. The usual CLT also requires finite positive variance and gives approximate normal sample means with standard error $\sigma/\sqrt n$.
• Hypothesis testing: specify $H_0$, pick a significance level $\alpha$, compute a test statistic, compare to the critical value or p-value. Reject $H_0$ when p-value $< \alpha$.
• Type I error = reject true $H_0$ (probability $\alpha$, the size). Type II error = fail to reject false $H_0$ (probability $\beta$). Power = $1 - \beta$, and lowering $\alpha$ raises $\beta$.

Exam shortcut

When a question gives a test statistic, p-value, and $\alpha$, the decision is mechanical: reject if p < $\alpha$ or |stat| > critical. The trap is interpretation: a small p-value is evidence against $H_0$, not the probability the null is true.

The full lesson (about 4,583 words, 31 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15

Browse all free FRM Part I lessons or jump into free FRM Part I practice questions.