Conformal Prediction in Hierarchical Classification

2025-01-31

AI Summary

To address the challenge in hierarchical classification where conformal prediction struggles to simultaneously guarantee statistical validity and predictive accuracy, this paper introduces HierCP—the first split-conformal prediction framework adapted to tree-structured label hierarchies. Methodologically: (1) it proposes a hierarchical calibration strategy based on internal nodes to ensure path consistency across the hierarchy; (2) it incorporates a representation complexity constraint, enabling derivation of a more compact and flexible prediction set generation algorithm. Experiments across multiple benchmark datasets demonstrate that HierCP strictly achieves the nominal coverage level while reducing average prediction set size by 23%–41% compared to state-of-the-art hierarchical conformal methods. This work provides the first theoretically grounded yet practically effective conformal prediction solution for trustworthy hierarchical classification.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Constraint Programming
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Conformal prediction has emerged as a widely used framework for constructing valid prediction sets in classification and regression tasks. In this work, we extend the split conformal prediction framework to hierarchical classification, where prediction sets are commonly restricted to internal nodes of a predefined hierarchy, and propose two computationally efficient inference algorithms. The first algorithm returns internal nodes as prediction sets, while the second relaxes this restriction, using the notion of representation complexity, yielding a more general and combinatorial inference problem, but smaller set sizes. Empirical evaluations on several benchmark datasets demonstrate the effectiveness of the proposed algorithms in achieving nominal coverage.

Problem

• Hierarchical Classification
• Conformal Prediction
• Consistent Prediction

Innovation

• Conformal Prediction
• Hierarchical Classification
• Efficiency Improvement