Generalized Hierarchical Conformal Prediction

2026-08-15

AI Summary

This study addresses the failure of standard stratified conformal prediction in small-sample grouped data settings caused by the absence of symmetry. To overcome this limitation, we propose Generalized Hierarchical Conformal Prediction (GHCP), which innovatively incorporates a randomized donation mechanism and a constrained filtering strategy. By dynamically adjusting reference group sizes to restore exchangeability and effectively leveraging initial observations to optimize nonconformity scores, GHCP enhances inference reliability. Experiments on both simulated datasets and the American Community Survey demonstrate that GHCP significantly improves predictive performance in small-sample scenarios while achieving valid distribution-free inference. Consequently, this method resolves the bottleneck of inefficient initial sample utilization inherent in existing approaches, providing a robust solution for grouped data analysis under limited sample conditions.

Abstract

Many prediction problems arise with data collected in groups. In this setting, hierarchical conformal prediction (HCP) (Lee et al., 2026) provides distribution-free prediction sets for a new observation from a previously unseen group under hierarchical exchangeability. In many applications, however, prediction is conducted only after a few observations from the group of interest have already been collected. Standard HCP cannot leverage these observations, as its required symmetry conditions do not hold in this setting. At the same time, the initial sample may still be too small for standard conformal prediction applied within the test group to be informative. We develop predictive inference methods for this setting. Our proposed method, Generalized HCP (GHCP), restores the relevant symmetry needed for conformal inference by assigning the test group a randomly "donated" reference group size. GHCP further leverages the initial test group observations to improve the quality of the nonconformity scores for prediction within that group. To improve efficiency, we introduce a variant that restricts the set of eligible donors. We demonstrate the performance of the proposed method through simulations and an illustration on the American Community Survey dataset.

Problem

• Hierarchical Conformal Prediction
• Grouped Data
• Prediction Sets
• Exchangeability

Innovation

• Generalized Hierarchical Conformal Prediction
• Symmetry Restoration
• Donated Reference Group Size
• Nonconformity Score Optimization
• Hierarchical Exchangeability