Perform model selection using graphical procedures, hypothesis tests (Kolmogorov-Smirnov, Chi-square goodness-of-fit, Likelihood ratio), and score-based approaches (SBC, BIC, AIC).

You fit five severity models. Three pass a chi-square test, two pass KS, and Akaike information criterion (AIC) and BIC disagree. You have ten minutes. The decision tree below is what you need.

Preliminary investigation comes before any formal test. Treat model selection as a five-step pipeline, not a single hypothesis test. Skipping the early steps is how you end up running a chi-square on data that obviously needs a mixture.

• Step (i): raw-data inspection. Plot a histogram, compute the sample mean, variance, skewness, and kurtosis. A sample variance equal to the mean suggests Poisson.
• Step (ii): empirical CDF or Kaplan-Meier. With complete data use $F_n(x)$ directly. With left-truncation at $d$ or right-censoring at $u$, build the Kaplan-Meier (or Nelson-Aalen) estimator so the comparison curve...
• Step (iii): graphical procedures. Covered below.
• Step (iv): hypothesis tests. Covered below.
• Step (v): score-based criteria. Covered below.

KEY: Steps (i) and (ii) DIAGNOSE the data; steps (iii)-(v) JUDGE the model. Run them in order.

Common mistakes

• Running KS on grouped data. KS needs individual observations; use chi-square instead.
• Running KS on a discrete distribution (Poisson, geometric, negative binomial). KS and Anderson-Darling apply only to continuous distributions.
• Forgetting to subtract $r$ from chi-square df. The df is $k-1-r$, not $k-1$, whenever parameters were estimated from this sample.

Bottom line

• KS statistic: $D={\max }_x|F_n(x)-F^{\ast }(x)|$. Continuous distributions and individual data only, never grouped. Critical value falls as $n$ grows.
• Chi-square GoF: $Q=\sum (O_j-E_j{)}^2/E_j$, df $=k-1-r$ ($r$ = parameters estimated from the data). Merge tail groups until every $E_j\ge 5$.
• Likelihood ratio: $2({\ell }_1-{\ell }_0)\sim {\chi }_{\Delta r}^2$. Nested models only. Reject the simple model when the statistic exceeds the critical value.
• AIC $=-2\ell +2r$; BIC/SBC $=-2\ell +r\ln n$. Smaller is better. BIC penalizes parameters harder once $n>7$.

Exam shortcut

For KS at 5%, the critical value is roughly $1.36/\sqrt{n}$. At $n=100$ that is 0.136; at $n=400$ it is 0.068. BIC beats AIC for parsimony once $\ln n>2$, so for any $n\ge 8$ expect BIC to favor the simpler model. For two nested candidates with $\Delta r=1$, the LRT critical value at 5% is 3.84. Any $2\Delta \ell >3.84$ means the extra parameter earns its keep.

The full lesson (about 5,320 words, 35 min read) adds 5 worked examples, all 12 common mistakes, a self-check, free in the app.

Learning objectives

• 4e

Browse all free Exam ASTAM lessons or jump into free Exam ASTAM practice questions.