Perform the computations behind clustering procedures (e.g., K-means, hierarchical).

A claims-severity team wants to segment 50,000 policyholders by behavior without using any labeled outcome. Clustering is unsupervised: no $Y$, just structure in $X$.

Why two families. K-means is fast and scales to millions of points but requires $K$ and is sensitive to initialization. Hierarchical produces a full tree you can cut at any level, but is $O(n^2)$ in memory and gets brittle past about 10,000 points. The 50,000-policyholder segmentation from the opener therefore goes to K-means: the hierarchical pairwise matrix would need roughly 2.5 billion entries, well past the memory wall.

WCSS measures how tight each cluster is: small squared distances from each point to its centroid mean the cluster is a coherent group, while large distances mean members are scattered. Given $n$ points $x_1,\dots ,x_n\in {\mathbb{R}}^p$ and a target number of clusters $K$, K-means partitions points into clusters $C_1,\dots ,C_K$ to minimize:

$$\text{WCSS}=\sum _{k=1}^K\sum _{i\in C_k}\Vert x_i-{\bar{x}}_k{\Vert }^2$$

where ${\bar{x}}_k=\frac{1}{|C_k|}{\sum }_{i\in C_k}{x}_i$ is the centroid.

Common mistakes

• Treating K-means convergence as global optimality. Lloyd's algorithm finds a local minimum. Reporting WCSS from a single random start understates the true minimum; the canonical fix is 20 or more restarts with different initializations.
• Forgetting to standardize. With features in raw units, Euclidean distance is dominated by the largest-variance feature. The wrong-answer pattern is a clustering driven entirely by one variable.
• Using K-means on categorical features. Means of categorical codes are nonsense. Use K-modes or a Gower-distance hierarchical clustering instead.

Bottom line

• K-means minimizes total within-cluster sum of squares (WCSS), ${\sum }_k{\sum }_{i\in C_k}\Vert x_i-{\bar{x}}_k{\Vert }^2$, using squared Euclidean distance. You pick $K$ up front.
• Lloyd's algorithm alternates: assign each point to its nearest centroid, then recompute centroids. WCSS strictly decreases each iteration and converges to a local (not global) optimum.
• Choose $K$ with the elbow method on WCSS; minimum WCSS alone fails because WCSS hits zero at $K=n$.
• Hierarchical clustering (agglomerative) starts with $n$ singletons and merges the two closest clusters at each step, producing a dendrogram whose height records each merge. No $K$ needed in advance.

Exam shortcut

If the problem gives $K$ and starting centroids, expect a K-means trace: do one or two iterations of assign-then-update, then check whether assignments stabilized. If the problem gives a pairwise distance matrix and asks for merge order, it is agglomerative hierarchical: find the smallest entry, merge, update with the named linkage rule, repeat.

The full lesson (about 2,083 words, 14 min read) adds 2 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• C7

Browse all free MAS-II lessons or jump into free MAS-II practice questions.