Estimate a sufficient statistic for a distribution.

A sufficient statistic compresses an entire sample into a few numbers without losing any information about the unknown parameter. Once you spot it, the MLE and the UMVUE almost write themselves.

Definition. $T(X_1,\ldots,X_n)$ is sufficient for $\theta$ if the conditional distribution of the sample given $T$ does not depend on $\theta$. Once $T$ is known, the raw data carry no extra information about $\theta$.

Factorization theorem. Checking conditional distributions directly is painful. The factorization theorem is the workhorse.

$$f(x_1,\ldots,x_n;\theta) = g\!\left(T(x_1,\ldots,x_n),\theta\right)\cdot h(x_1,\ldots,x_n)$$

Everything with $\theta$ goes into $g$. Everything that depends only on the data goes into $h$.

Exponential family shortcut. If the density takes the canonical form

$$f(x;\theta) = h(x)\,c(\theta)\exp\!\left(\sum_{j=1}^{k} w_j(\theta)\, t_j(x)\right)$$

then $\left(\sum_i t_1(X_i),\ldots,\sum_i t_k(X_i)\right)$ is sufficient, and (with open natural parameter space) also complete and minimal sufficient.

KEY: Read off $t_j(x)$ directly from the exponent. For Poisson, the exponent contains $x\ln\lambda$, so $t(x)=x$ and the sufficient statistic is $\sum X_i$.

Uniform is the exception. Uniform(0,$\theta$) is not exponential family. Its sufficient statistic comes from the indicator: $\prod \mathbb{1}\{x_i<\theta\}=\mathbb{1}\{\max x_i<\theta\}$. The dependence on $\theta$ collapses onto the maximum, so $T=\max X_i$ is sufficient.

Common mistakes

• Treating $\bar X$ and $\sum X_i$ as different statistics. They differ by a known constant $n$, so each is a one-to-one function of the other. Either qualifies.
• Letting $\theta$ sneak into $h(x)$. Indicators like $\mathbb{1}\{x_i<\theta\}$ depend on $\theta$ and belong in $g$. Misplacing them breaks the factorization and the conclusion.
• Claiming $\bar X$ is sufficient for Uniform(0,$\theta$). It is not. The correct statistic is $\max X_i$, which captures the boundary information that the mean throws away.

Bottom line

• Sufficiency holds when the conditional distribution of the sample given $T$ is parameter-free, so $T$ retains all information about $\theta$.
• Fisher-Neyman factorization: $T$ is sufficient for $\theta$ iff the joint density factors as $g(T(x),\theta)\cdot h(x)$, where $h$ does not depend on $\theta$.
• Exponential family rule: if $f(x;\theta)=h(x)c(\theta)\exp(\sum w_j(\theta)t_j(x))$, then $\left(\sum_i t_j(X_i)\right)$ is sufficient by inspection.
• One-parameter benchmarks: $\sum X_i$ for Bernoulli, Poisson, Exponential, Geometric, and Normal($\mu,\sigma^2$ known); $\max X_i$ for Uniform(0,$\theta$).

Exam shortcut

Spot the exponential family form first. If the density factors as $h(x)\,c(\theta)\exp(w(\theta)\,t(x))$, then $\sum t(X_i)$ is sufficient without any further algebra. For bounded-support families, look at indicator functions: the bound that depends on $\theta$ becomes the sufficient statistic, usually the maximum or minimum order statistic. For UMVUE questions, do not compute $E[W\mid T]$ from scratch; guess a linear rescaling of the complete sufficient statistic that is unbiased.

The full lesson (about 2,239 words, 15 min read) adds 5 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• B2

Browse all free MAS-I lessons or jump into free MAS-I practice questions.