Silhouette Score

Judging Clusters Without Labels

With supervised models we compare predictions to the true labels. In clustering there are no labels, so we need a different question: are points close to their own cluster and far from the others?

The Formula

For each point $i$:

• $a(i)$ = average distance to the other points in its own cluster (how tight its cluster is).
• $b(i)$ = average distance to the points in the nearest other cluster (how far the neighbours are).

The silhouette score of a clustering is the average $s(i)$ over all points.

By Hand

Study hours from the K-means lesson: 1, 2, 3 in cluster A and 10, 11, 12 in cluster B.

Point 1 (in A):

• $a=\frac{|1-2|+|1-3|}{2}=\frac{1+2}{2}=1.5$
• $b=\frac{|1-10|+|1-11|+|1-12|}{3}=\frac{9+10+11}{3}=10$
• $s=\frac{10-1.5}{10}=0.85$

Point 2 (in A):

• $a=\frac{1+1}{2}=1$
• $b=\frac{8+9+10}{3}=9$
• $s=\frac{9-1}{9}\approx 0.889$

Point 3 is the mirror image of point 1 on the near side: $a=1.5$, $b=8$, $s=\frac{6.5}{8}=0.8125$. Cluster B is symmetrical, so the overall average is about 0.85: a clean clustering.

import numpy as np
from sklearn.metrics import silhouette_samples, silhouette_score

hours = np.array([[1], [2], [3], [10], [11], [12]])
labels = np.array([0, 0, 0, 1, 1, 1])

print("each point:", silhouette_samples(hours, labels).round(3))
print(f"average   : {silhouette_score(hours, labels):.3f}")

bad = np.array([0, 0, 1, 1, 0, 1])      # a deliberately poor grouping
print(f"bad grouping average: {silhouette_score(hours, bad):.3f}")

each point: [0.85  0.889 0.812 0.812 0.889 0.85 ]
average   : 0.850
bad grouping average: -0.038

A poor grouping scores far lower, with some points negative.

Choosing K

Unlike inertia, the silhouette score doesn't automatically improve with more clusters. Splitting a real cluster in two makes points close to the "other" half, which lowers $b$. So the K with the highest score is a strong candidate.

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs

X, _ = make_blobs(n_samples=300, centers=4, cluster_std=0.9, random_state=1)

for k in range(2, 8):
    labels = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(X)
    print(f"K={k}: silhouette {silhouette_score(X, labels):.3f}")

K=2: silhouette 0.715
K=3: silhouette 0.606
K=4: silhouette 0.661
K=5: silhouette 0.576
K=6: silhouette 0.519
K=7: silhouette 0.399

We generated 4 groups, yet the highest score is at K = 2. Look at the centroids from the K-means lesson: three of the four blobs sit close together on one side, and one sits far away. At a coarse level, "far blob vs the rest" really is the cleanest split. K = 4 is a clear second peak, and it's where the elbow pointed.

The lesson: no single score settles K. Check the elbow, the silhouette and, if you can, a plot of the clusters, then choose the K that makes sense for your problem.

Summary

• $a$ = average distance within the own cluster; $b$ = to the nearest other cluster.
• $s=\frac{b-a}{\max (a,b)}$, from −1 (wrong cluster) to 1 (perfect).
• Average over points to score a clustering; a high average marks a good K, but check it against the elbow and a plot.
• Next: a clustering method that finds any shape and flags outliers, DBSCAN.