25 / (2 * log(2))[1] 18.033697 Self-organising maps
Study time: about 3 hours. Theory only — no coding in the exam. Largest DR question bank.
7.1 Pass
7.1.1 Two spaces
A SOM places a fixed grid of nodes in map space (usually 2-D). Each node j also owns a prototype \mathbf{m}_j in data space (same dimension p as the features).
- Grid location \boldsymbol{\ell}_j: fixed display coordinates.
- Prototype \mathbf{m}_j: learnt p-vector describing what that node represents.
An observation is mapped to its best matching unit (BMU): the node whose prototype is closest in data space.
This “neighbourhood” lives on the grid. An ISOMAP neighbourhood lives among data points. Same word, different geometry.
7.1.2 Why a grid helps
The plotting axes are fixed by the grid. An outlier does not stretch a displayed axis the way it can in PCA. Large variance differences between variables also cannot directly stretch map axes. Movement across the grid is not a single linear combination of variables (unlike a PCA axis): different regions can change different features.
7.1.3 Update
When observation \mathbf{x}_i has BMU b_i, every node j moves toward \mathbf{x}_i:
\Delta\mathbf{m}_j = \alpha_t\, h_{b_i j}(t)\,(\mathbf{x}_i-\mathbf{m}_j).
- \alpha_t: learning rate (overall step size).
- h_{b_i j}(t): neighbourhood weight — large when j is near b_i on the grid; must be non-negative (otherwise prototypes would be pushed away from the data).
Early training: larger \alpha_t and wider neighbourhood → broad organisation. Later: both shrink → local refinement.
A node with no assigned observations still has a prototype: neighbours’ BMUs pull it. Empty nodes arise when the grid is larger than the data need, or when surrounding prototypes “average out” a location.
7.1.4 Worked update (question bank)
h_{b_i j}(t)=\exp\!\Bigl(-\frac{\|\boldsymbol{\ell}_{b_i}-\boldsymbol{\ell}_j\|^2}{2\sigma^2}\Bigr), \quad \alpha_t=0.5, \quad \mathbf{x}_i-\mathbf{m}_j=\begin{pmatrix}0.4\\-0.1\end{pmatrix}, \quad \boldsymbol{\ell}_{b_i}=\begin{pmatrix}1\\2\end{pmatrix}, \quad \boldsymbol{\ell}_j=\begin{pmatrix}5\\5\end{pmatrix}.
Target change: (0.1,-0.025)'.
From the first component: 0.1=0.5\cdot h\cdot 0.4\Rightarrow h=0.5.
\|\boldsymbol{\ell}_{b_i}-\boldsymbol{\ell}_j\|^2=(1-5)^2+(2-5)^2=25.
\exp(-25/(2\sigma^2))=0.5\Rightarrow\sigma^2=25/(2\ln 2)\approx 18.03.
7.1.5 SOM and k-means
Both use prototypes. K-means updates a centroid using only its own members. SOM prototypes are also pulled by neighbouring BMUs — a restricted k-means that cannot match within-node data as freely because topology is preserved.
7.1.6 Reading a fitted map
- Component plane: colour each node by one coordinate of its prototype (one original variable).
- U-matrix: show data-space distances between neighbouring prototypes; large values suggest boundaries.
- Quantisation error: average distance from each observation to its BMU (fit in data space).
- Topographic error: e.g. proportion of observations whose first and second BMUs are not adjacent on the grid (fit of the topology).
The two errors complement each other: good quantisation with bad topography (or the reverse) is an incomplete story.
7.1.7 Efficiency
SOM compares observations to a fixed number of prototypes. It need not store all n(n-1)/2 pairwise observation dissimilarities (unlike MDS).
7.1.8 Contrast with ISOMAP nodes
SOM node: learnt prototype + fixed grid slot; many observations may map to it.
ISOMAP graph node: one actual observation.
7.2 Reference
- Training progress: monitor quantisation error and neighbourhood radius over iterations.
- Do not read a SOM as a PCA biplot: there is no single linear meaning to “moving right on the map”.
- Non-linearity: ISOMAP changes the distance then embeds continuously; SOM typically uses ordinary data-space distance onto a discrete grid.
7.3 Recall
Official — empty node prototype
Does an empty node have a prototype? How?
Answer
Yes. Initialised, then updated when neighbouring nodes are BMUs.
Official — BMU
What is the best matching unit?
Answer
The node whose prototype is closest to the observation in data space.
Official — \sigma^2 calculation
With the bank’s numbers, \sigma^2?
Answer
\approx 18.03, from h=0.5 and \|\Delta\boldsymbol{\ell}\|^2=25.
Official — quantisation vs topographic
Define both and how they complement.
Answer
QE: average distance to BMU. TE: how often first and second BMUs are non-adjacent. One scores data-space fit; the other scores grid topology.
Official — restricted k-means
In what sense is a SOM restricted k-means?