Post Hoc Regression Refinement via Pairwise Rankings

2025-08-22

AI Summary

To address the degradation of prediction accuracy in deep regression models under data-scarce regimes, this paper proposes a plug-and-play post-processing method that leverages pairwise ranking information—obtained either from large language models or domain experts—to calibrate base regressor outputs. Specifically, it fuses the original regression predictions with a ranking-driven estimate via inverse-variance weighting, requiring neither model retraining nor fine-tuning. This work is the first to incorporate pairwise ranking into regression post-processing, yielding a model-agnostic and training-free framework. Empirical evaluation on molecular property prediction tasks demonstrates that as few as 20 pairwise comparisons suffice to achieve up to a 10% relative reduction in mean absolute error, validating the method’s effectiveness, low annotation overhead, and strong cross-task generalizability.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Preferences

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Accurate prediction of continuous properties is essential to many scientific and engineering tasks. Although deep-learning regressors excel with abundant labels, their accuracy deteriorates in data-scarce regimes. We introduce RankRefine, a model-agnostic, plug-and-play post hoc method that refines regression with expert knowledge coming from pairwise rankings. Given a query item and a small reference set with known properties, RankRefine combines the base regressor's output with a rank-based estimate via inverse variance weighting, requiring no retraining. In molecular property prediction task, RankRefine achieves up to 10% relative reduction in mean absolute error using only 20 pairwise comparisons obtained through a general-purpose large language model (LLM) with no finetuning. As rankings provided by human experts or general-purpose LLMs are sufficient for improving regression across diverse domains, RankRefine offers practicality and broad applicability, especially in low-data settings.

Problem

• Refining regression predictions using pairwise rankings
• Improving accuracy in data-scarce scientific prediction tasks
• Leveraging expert knowledge without model retraining requirements

Innovation

• Post hoc regression refinement via pairwise rankings
• Model-agnostic plug-and-play method requiring no retraining
• Combines base regressor with rank-based estimate via inverse weighting