FOSC-X: An Extended Framework for Optimal Local Cuts and Non-Horizontal Cluster Selection from Clustering Hierarchies

2026-06-17

AI Summary

Traditional approaches typically extract only a single optimal flat clustering from a hierarchical clustering tree, thereby failing to uncover the diverse latent structures inherent in data. To address this limitation, this work proposes FOSC-X, a novel framework that, for the first time, enables efficient generation of the top-M globally optimal and mutually distinct flat clusterings in linear time, with or without a constraint on the number of clusters. The method leverages dynamic programming and exploits the composability of locally optimal solutions within subtrees, while introducing feasibility-based upper and lower bounds to prune invalid or dominated candidates. Experimental results demonstrate that FOSC-X not only automatically discovers multiple high-quality alternative clustering structures but also achieves an effective balance between computational efficiency and solution optimality.

Abstract

Extracting a flat clustering solution from a hierarchy is a common task in practical cluster analysis and can be formulated as an optimisation problem. Existing approaches focus on finding a single optimal solution. We introduce FOSC-X, a framework for extracting the top-M globally optimal flat clusterings from local, non-horizontal cuts of a hierarchical cluster tree, while optionally enforcing constraints on the number of clusters. This enables automatic identification of multiple high-quality alternative clusterings that capture different aspects of the hierarchical structure. Without constraints, the top-M problem can be solved in polynomial time using dynamic programming, exploiting the property that locally optimal partial candidates within subtrees can be combined to form globally optimal solutions while automatically determining the number of clusters. However, this can lead to solutions with numbers of clusters that are ultimately undesirable -- e.g., too large to be meaningful or practically analysed within a particular application domain. Imposing cluster-count constraints breaks the optimality property underlying the unconstrained dynamic programming approach, since locally optimal partial candidates may no longer combine into feasible globally optimal solutions. FOSC-X addresses this challenge through a dynamic programming strategy that maintains compact sets of feasible candidates using lower and upper feasibility bounds while pruning infeasible or dominated combinations. The resulting method guarantees optimal rankings of the top-M solutions with linear-time complexity in the number of cluster nodes and dataset size, both with and without cluster-count constraints. Experiments show that FOSC-X efficiently reveals alternative clustering structures overlooked by single-solution extraction methods.

Problem

• hierarchical clustering
• flat clustering extraction
• cluster selection
• optimization
• cluster constraints

Innovation

• FOSC-X
• non-horizontal cuts
• top-M optimal clusterings
• dynamic programming with constraints
• hierarchical clustering extraction