Types of Modeling Problems and Methods

Every SRM problem starts with the same triage: do you have a labeled target, and if so, is that target numeric or categorical? Get this wrong and you pick the wrong toolbox. This first lesson is the syllabus's vocabulary map, so it names methods before it builds them; the pointers to where each one is taught sit just above the methods list below.

In supervised learning you observe both predictors $X_1,\dots ,X_p$ and a response $Y$ for each training observation. The goal is a function $\hat{f}$ so that $\hat{Y}=\hat{f}(X)$ on new data. In unsupervised learning you observe only $X$; there is no labeled $Y$. The goal is structure: groups (clustering), low-dimensional summaries from principal component analysis (PCA), which compresses correlated inputs into a smaller set of axes, or association rules.

KEY: "Supervised" means a teacher (the label Y) tells the model when it is right. No label = no supervision = unsupervised.

Semi-supervised learning sits in between: most observations are unlabeled, a few are labeled. It exploits the cheap unlabeled inputs to sharpen the supervised fit.

Common mistakes

• Calling logistic regression a regression method. The response is categorical, so the task is classification. The name is historical.
• Treating ordinal categorical Y as numeric. Risk class A/B/C looks orderable but the gaps are not equal. Forcing regression imposes a false numeric scale.
• Confusing clustering with classification. Clustering has no labels; classification has labels and assigns new observations to known classes. If someone says "unsupervised classification" they almost always mean clustering.

Bottom line

• Supervised learning has a labeled target Y; unsupervised learning has only inputs X. Clustering and PCA are unsupervised; regression and classification are supervised.
• Regression predicts a quantitative (numeric) response; classification predicts a qualitative (categorical) response. The response type, not the method name, fixes the task.
• K-nearest neighbors, trees, and neural nets do BOTH regression and classification. Linear regression is regression-only; logistic regression is classification-only despite the name.
• Inference asks "how does X affect Y?"; prediction asks "what is $\hat{Y}$?" Prediction tolerates black boxes; inference demands interpretable coefficients.

Exam shortcut

If the prompt mentions "labels," "target," or a specific Y column, it is supervised; the only question left is regression vs. classification. If the prompt says "group," "segment," "structure," or "patterns" with no Y, it is unsupervised. Method names ending in "regression" are usually regression (linear, Poisson, ridge), with logistic as the lone classification exception.

The full lesson (about 1,413 words, 9 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• 1a

Browse all free Exam SRM lessons or jump into free Exam SRM practice questions.