Perform the computations behind clustering procedures (e.g., K-means, hierarchical).

A claims-severity team wants to segment 50,000 policyholders by behavior without using any labeled outcome. Clustering is unsupervised: no $Y$, just structure in $X$.

Why two families. K-means is fast and scales to millions of points but requires $K$ and is sensitive to initialization. Hierarchical produces a full tree you can cut at any level, but is $O(n^2)$ in memory and gets brittle past about 10,000 points. The 50,000-policyholder segmentation from the opener therefore goes to K-means: the hierarchical pairwise matrix would need roughly 2.5 billion entries, well past the memory wall.

WCSS measures how tight each cluster is: small squared distances from each point to its centroid mean the cluster is a coherent group, while large distances mean members are scattered. Given $n$ points $x_1,\dots,x_n \in \mathbb{R}^p$ and a target number of clusters $K$, K-means partitions points into clusters $C_1,\dots,C_K$ to minimize:

$$\text{WCSS} = \sum_{k=1}^{K}\sum_{i\in C_k}\|x_i - \bar{x}_k\|^{2}$$

where $\bar{x}_k = \frac{1}{|C_k|}\sum_{i\in C_k} x_i$ is the centroid.

Common mistakes

• Treating K-means convergence as global optimality. Lloyd's algorithm finds a local minimum. Reporting WCSS from a single random start understates the true minimum; the canonical fix is 20 or more restarts with different initializations.
• Forgetting to standardize. With features in raw units, Euclidean distance is dominated by the largest-variance feature. The wrong-answer pattern is a clustering driven entirely by one variable.
• Using K-means on categorical features. Means of categorical codes are nonsense. Use K-modes or a Gower-distance hierarchical clustering instead.

Bottom line

• K-means minimizes total within-cluster sum of squares (WCSS), $\sum_k \sum_{i\in C_k}\|x_i-\bar{x}_k\|^2$, using squared Euclidean distance. You pick $K$ up front.
• Lloyd's algorithm alternates: assign each point to its nearest centroid, then recompute centroids. WCSS strictly decreases each iteration and converges to a local (not global) optimum.
• Choose $K$ with the elbow method on WCSS; minimum WCSS alone fails because WCSS hits zero at $K=n$.
• Hierarchical clustering (agglomerative) starts with $n$ singletons and merges the two closest clusters at each step, producing a dendrogram whose height records each merge. No $K$ needed in advance.

Exam shortcut

If the problem gives $K$ and starting centroids, expect a K-means trace: do one or two iterations of assign-then-update, then check whether assignments stabilized. If the problem gives a pairwise distance matrix and asks for merge order, it is agglomerative hierarchical: find the smallest entry, merge, update with the named linkage rule, repeat.

The full lesson (about 2,807 words, 19 min read) adds 8 worked examples, all 6 common mistakes, a self-check, free in the app.

Learning objectives

• C7

Browse all free MAS-II lessons or jump into free MAS-II practice questions.