x1 <- c(1, 1)
x2 <- c(4, 5)
c(
euclidean = sqrt(sum((x1 - x2)^2)),
manhattan = sum(abs(x1 - x2))
)euclidean manhattan
5 7 1 Shared clustering ideas
Study time: about 1 hour. No algorithm yet — only the language every later clustering chapter uses.
1.1 Pass
1.1.1 What clustering is trying to do
You have unlabelled observations. Clustering forms groups so that observations in the same cluster are relatively similar, and observations in different clusters are relatively dissimilar.
That sentence hides a modelling choice: you define what “similar” means. Change the features or the dissimilarity, and the groups change.
Three words recur:
- Observation: the object being grouped (a state, a route, a clinic).
- Feature: a measured property used to compare observations.
- Dissimilarity: a number saying how different two observations are. Small means similar; large means dissimilar.
1.1.2 Cohesion and separation
A useful clustering usually aims for:
- Cohesion: observations within a cluster are close to one another.
- Separation: observations in different clusters are far apart.
Many algorithms turn this into “minimise within-cluster dissimilarity” (and sometimes also “maximise between-cluster dissimilarity”). The silhouette later scores both ideas for a single observation.
1.1.3 Dissimilarity and the matrix
Write the dissimilarity between observations i and i' as
d_{ii'} = d(\mathbf{x}_i, \mathbf{x}_{i'}).
Collect all pairs in an n \times n matrix \mathbf{D} with zeros on the diagonal and d_{ii'} = d_{i'i}. Some algorithms (hierarchical clustering) can work from \mathbf{D} alone. K-means works from the feature matrix and an implied squared Euclidean distance.
1.1.4 Euclidean and Manhattan
For two observations with p numerical features,
d_E(\mathbf{x}_i, \mathbf{x}_{i'}) = \sqrt{\sum_{j=1}^{p}(x_{ij}-x_{i'j})^2} \qquad \text{(straight-line distance)},
d_M(\mathbf{x}_i, \mathbf{x}_{i'}) = \sum_{j=1}^{p}|x_{ij}-x_{i'j}| \qquad \text{(distance travelled along the axes)}.
Standard k-means uses squared Euclidean distance, not Manhattan.
Worked pair. \mathbf{x}_1 = (1,1), \mathbf{x}_2 = (4,5).
d_E = \sqrt{(4-1)^2+(5-1)^2} = \sqrt{9+16} = 5, \qquad d_M = |4-1|+|5-1| = 7.
The two numbers differ because one squares and roots the gaps, while the other adds absolute gaps. Neither is “wrong”; they answer different geometric questions.
1.1.5 Feature scale
If one feature is recorded in much larger units, it dominates the total distance. Distances between the same observations change when relative scales change.
Standardisation is a modelling choice, not an automatic default. Use it when comparable numerical influence matches the purpose of the analysis. Leave scales alone when the units themselves carry the meaning you care about (for example crime counts that should keep their absolute magnitudes).
par(mfrow = c(1, 2))
A <- matrix(c(0, 0, 1, 1), 2, 2, byrow = TRUE)
plot(A, pch = 19, xlim = c(-0.2, 1.2), ylim = c(-0.2, 1.2),
main = "Equal scales", xlab = "x1", ylab = "x2")
segments(A[1, 1], A[1, 2], A[2, 1], A[2, 2])
B <- A
B[, 2] <- B[, 2] * 10
plot(B, pch = 19, main = "x2 × 10", xlab = "x1", ylab = "x2")
segments(B[1, 1], B[1, 2], B[2, 1], B[2, 2])
1.1.6 Data type and dissimilarity (reference table)
| Data type | Possible dissimilarity | Important question |
|---|---|---|
| Continuous numerical | Euclidean or Manhattan | Are scale differences meaningful? |
| Nominal categories | Simple matching | Are all mismatches equally different? |
| Asymmetric binary | Jaccard | Should shared absences count? |
| Mixed types | Gower | How should types be combined? |
| Text / embeddings | Cosine | Does direction matter more than magnitude? |
This exam’s coding uses numerical features and Euclidean distance. The table is for theory questions that ask whether a measure is appropriate.
1.1.7 What is not yet defined
The silhouette needs a fitted partition. It appears in the k-means chapter after Lloyd’s algorithm has produced clusters.
1.2 Reference
- Clustering is unsupervised: known labels are not supplied to the algorithm.
- The same observations can be grouped differently (two broad groups, four local groups, nested groups). Clusters are constructions; there is often no single “correct” clustering.
- Algorithm choice matters because algorithms differ in the process used to build assignments. Different algorithms can share the same dissimilarity.
- A practical workflow frames the question, prepares the features and dissimilarity, fits candidates, shortlists K, then judges stability, profiles clusters, and decides whether anyone should use the groups. Weak evidence sends you back to preparation and fitting.
1.3 Recall
Original — distance choice
Two clinics are compared on age and blood pressure. Blood pressure is recorded in mmHg (roughly 80–180) and age in years (roughly 20–90). Without scaling, which feature tends to dominate Euclidean distance, and why?
Answer
Blood pressure: its gaps are larger in absolute units, so they contribute more to \sum_j (x_{ij}-x_{i'j})^2. Standardise if equal influence is intended.
Official-style — compute both distances
For \mathbf{x}_1=(1,1) and \mathbf{x}_2=(4,5), state d_E and d_M.