Diversifying Conformal Selections

2025-06-19

AI Summary

This work addresses the candidate set selection problem—arising in applications such as drug discovery and hiring—where both high confidence (i.e., strict false discovery rate control) and high diversity are simultaneously required. We propose Diversity-Aware Conformal Selection (DACS), the first model-agnostic framework that provably controls the false discovery rate (FDR ≤ α) while guaranteeing diversity. DACS innovatively integrates conformal e-values, optimal stopping theory, and combinatorial optimization; it employs a diversity metric to guide adaptive stopping decisions, supporting both exact optimization and efficient heuristic algorithms. Experiments on synthetic data, molecular screening benchmarks, and real-world hiring datasets demonstrate that DACS consistently maintains FDR well below the target threshold (α = 0.1) while improving diversity by 32% over state-of-the-art baselines—establishing new performance trade-offs between statistical reliability and representational richness.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

When selecting from a list of potential candidates, it is important to ensure not only that the selected items are of high quality, but also that they are sufficiently dissimilar so as to both avoid redundancy and to capture a broader range of desirable properties. In drug discovery, scientists aim to select potent drugs from a library of unsynthesized candidates, but recognize that it is wasteful to repeatedly synthesize highly similar compounds. In job hiring, recruiters may wish to hire candidates who will perform well on the job, while also considering factors such as socioeconomic background, prior work experience, gender, or race. We study the problem of using any prediction model to construct a maximally diverse selection set of candidates while controlling the false discovery rate (FDR) in a model-free fashion. Our method, diversity-aware conformal selection (DACS), achieves this by designing a general optimization procedure to construct a diverse selection set subject to a simple constraint involving conformal e-values which depend on carefully chosen stopping times. The key idea of DACS is to use optimal stopping theory to adaptively choose the set of e-values which (approximately) maximizes the expected diversity measure. We give an example diversity metric for which our procedure can be run exactly and efficiently. We also develop a number of computational heuristics which greatly improve its running time for generic diversity metrics. We demonstrate the empirical performance of our method both in simulation and on job hiring and drug discovery datasets.

Problem

• Ensuring diverse candidate selection while maintaining high quality
• Controlling false discovery rate in model-free diverse selections
• Optimizing selection diversity using conformal e-values and stopping times

Innovation

• Diversity-aware conformal selection (DACS) method
• Optimal stopping theory for adaptive e-values
• Computational heuristics for efficiency