How to evaluate clustering with ground truth?

2026-06-25

AI Summary

Despite the availability of ground-truth labels, systematic guidance on effectively evaluating clustering quality remains limited. This work presents a comprehensive comparison of external clustering evaluation metrics based on set matching, explicitly distinguishing between cluster-level and point-level assessment needs. It advocates for the use of the intuitive and interpretable Centroid Index (CI) for cluster-level evaluation, while highlighting the advantages of metrics such as the Pairwise Set Index (PSI) in point-level contexts. Through an integrated analysis of CI, PSI, clustering accuracy (ACC), and related measures, the study establishes a practical guideline for metric selection tailored to different granularity requirements. This approach substantially enhances the fairness and interpretability of clustering performance evaluation.

Abstract

External indexes can be used for cluster evaluation when ground truth is available. We review the most common external validity indexes focusing on set-matching-based measures. We recommend centroid index (CI), because it is an intuitive cluster-level measure with an explainable result. If we need a more fine-tuned, point-level measure, there are more choices. Pair-set index (PSI) provides a normalized score which is not biased by cluster sizes. If all points should matter equally, then clustering accuracy (ACC) or any other set-matching measure is suitable.

Problem

• clustering evaluation
• external validity indexes
• ground truth
• cluster validation

Innovation

• external validity index
• centroid index
• pair-set index
• clustering evaluation
• set-matching measure