Estimate a sufficient statistic for a distribution.

A sufficient statistic compresses an entire sample into a few numbers without losing any information about the unknown parameter. Once you spot it, the MLE and the UMVUE almost write themselves.

Definition. $T(X_1,\dots ,X_n)$ is sufficient for $\theta$ if the conditional distribution of the sample given $T$ does not depend on $\theta$. Once $T$ is known, the raw data carry no extra information about $\theta$.

Factorization theorem. Checking conditional distributions directly is painful. The factorization theorem is the workhorse.

$$f(x_1,\dots ,x_n;\theta )=g\left(T(x_1,\dots ,x_n),\theta \right)\cdot h(x_1,\dots ,x_n)$$

Everything with $\theta$ goes into $g$. Everything that depends only on the data goes into $h$.

Exponential family shortcut. If the density takes the canonical form

$$f(x;\theta )=h(x)c(\theta )\exp \left(\sum _{j=1}^k{w}_j(\theta )t_j(x)\right)$$

then $\left({\sum }_i{t}_1(X_i),\dots ,{\sum }_i{t}_k(X_i)\right)$ is sufficient, and (with open natural parameter space) also complete and minimal sufficient.

KEY: Read off $t_j(x)$ directly from the exponent. For Poisson, the exponent contains $x\ln \lambda$, so $t(x)=x$ and the sufficient statistic is $\sum X_i$.

Uniform is the exception. Uniform(0,$\theta$) is not exponential family. Its sufficient statistic comes from the indicator: $\prod 1\{x_i<\theta \}=1\{\max x_i<\theta \}$. The dependence on $\theta$ collapses onto the maximum, so $T=\max X_i$ is sufficient.

Common mistakes

• Treating $\bar{X}$ and $\sum X_i$ as different statistics. They differ by a known constant $n$, so each is a one-to-one function of the other. Either qualifies.
• Letting $\theta$ sneak into $h(x)$. Indicators like $1\{x_i<\theta \}$ depend on $\theta$ and belong in $g$. Misplacing them breaks the factorization and the conclusion.
• Claiming $\bar{X}$ is sufficient for Uniform(0,$\theta$). It is not. The correct statistic is $\max X_i$, which captures the boundary information that the mean throws away.

Bottom line

• Sufficiency holds when the conditional distribution of the sample given $T$ is parameter-free, so $T$ retains all information about $\theta$.
• Fisher-Neyman factorization: $T$ is sufficient for $\theta$ iff the joint density factors as $g(T(x),\theta )\cdot h(x)$, where $h$ does not depend on $\theta$.
• Exponential family rule: if $f(x;\theta )=h(x)c(\theta )\exp (\sum w_j(\theta )t_j(x))$, then $\left({\sum }_i{t}_j(X_i)\right)$ is sufficient by inspection.
• One-parameter benchmarks: $\sum X_i$ for Bernoulli, Poisson, Exponential, Geometric, and Normal($\mu ,{\sigma }^2$ known); $\max X_i$ for Uniform(0,$\theta$).

Exam shortcut

Spot the exponential family form first. If the density factors as $h(x)c(\theta )\exp (w(\theta )t(x))$, then $\sum t(X_i)$ is sufficient without any further algebra. For bounded-support families, look at indicator functions: the bound that depends on $\theta$ becomes the sufficient statistic, usually the maximum or minimum order statistic. For UMVUE questions, do not compute $E[W\mid T]$ from scratch; guess a linear rescaling of the complete sufficient statistic that is unbiased.

The full lesson (about 1,461 words, 10 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• B2

Browse all free MAS-I lessons or jump into free MAS-I practice questions.