Hierarchical Clustering

Hierarchical Clustering is an unsupervised learning technique used to group similar data points into clusters based on a hierarchy. Unlike K-Means, you don't need to predefine the number of clusters. Instead, the algorithm builds a tree-like structure (dendrogram) representing nested groupings of data.

There are two main types of hierarchical clustering:

• Agglomerative (bottom-up): Start with individual data points, merge clusters iteratively.
• Divisive (top-down): Start with all data in one cluster and split recursively.

Key Concepts

Dendrogram

A dendrogram is a tree-like diagram that records the sequences of merges or splits. Cutting the dendrogram at a chosen level yields the desired number of clusters.

Distance Metrics

Hierarchical clustering relies on a distance metric:

• Euclidean (default)
• Manhattan
• Cosine, etc.

Linkage Criteria

Defines how the distance between clusters is calculated:

• Single Linkage – Minimum distance between points of the clusters
• Complete Linkage – Maximum distance between points
• Average Linkage – Average distance between points
• Ward’s Method – Minimizes variance within clusters

Python Example: Agglomerative Clustering

from sklearn.datasets import make_blobs
from sklearn.cluster import AgglomerativeClustering
import matplotlib.pyplot as plt
import scipy.cluster.hierarchy as sch

# 1. Generate synthetic data
X, _ = make_blobs(n_samples=150, centers=3, cluster_std=0.60, random_state=42)

# 2. Plot dendrogram to visualize hierarchy
plt.figure(figsize=(8, 5))
dendrogram = sch.dendrogram(sch.linkage(X, method='ward'))
plt.title("Dendrogram")
plt.xlabel("Data Points")
plt.ylabel("Euclidean Distance")
plt.show()

# 3. Fit Agglomerative Clustering with 3 clusters
model = AgglomerativeClustering(n_clusters=3, affinity='euclidean', linkage='ward')
labels = model.fit_predict(X)

# 4. Visualize Clusters
plt.scatter(X[:, 0], X[:, 1], c=labels, cmap='viridis')
plt.title("Agglomerative Clustering (3 Clusters)")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.show()

Output- 

Explanation

• Step 1: Create data with known centers.
• Step 2: Plot a dendrogram to observe the natural cluster hierarchy.
• Step 3: Use AgglomerativeClustering with ward linkage.
• Step 4: Visualize the clustering results.

When to Use Hierarchical Clustering

• When the number of clusters is unknown.
• When you want to understand the data structure or nested groupings.
• For smaller datasets (does not scale well for very large data).

Advantages

• No need to predefine number of clusters.
• Produces a dendrogram for visual analysis.
• Can capture nested data groupings.

Limitations

• Computationally expensive (O(n²) time and space).
• Sensitive to noise and outliers.
• Final clusters are not always optimal due to early merge decisions (no reassignments).

Tips for Beginners

• Use dendrograms to visually decide on the number of clusters.
• Stick with ward linkage and euclidean distance for standard tasks.
• Normalize data if features have different scales.

Tips for Professionals

• For large datasets, use scikit-learn’s connectivity constraints to speed up clustering.
• Combine PCA with hierarchical clustering for better performance and visualization.
• Use hierarchical clustering as a preprocessing step to inform other models.

Summary

• Hierarchical clustering builds a hierarchy of clusters without needing to specify K.
• Dendrograms help visualize and select cluster groups.
• Best suited for small to medium datasets with clear hierarchical structures.