Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation

2026-04-16

AI Summary

This work addresses the limitations of traditional learning-to-rank methods, which rely on a single predefined metric and suffer from training instability due to non-differentiable objective functions, as well as limited generalization across other evaluation metrics. The authors propose a metric-agnostic, listwise learning-to-rank framework that constructs differentiable loss functions through smooth approximations of ranking operators. Notably, this approach introduces gradient boosting machines into listwise optimization for the first time. By decoupling the learning process from any specific ranking metric, the method achieves both efficient training and strong generalization performance. Experimental results demonstrate that it consistently outperforms state-of-the-art approaches across multiple information retrieval evaluation metrics while maintaining comparable computational efficiency.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Learning-to-Rank (LTR) is a supervised machine learning approach that constructs models specifically designed to order a set of items or documents based on their relevance or importance to a given query or context. Despite significant success in real-world information retrieval systems, current LTR methods rely on one prefix ranking metric (e.g., such as Normalized Discounted Cumulative Gain (NDCG) or Mean Average Precision (MAP)) for optimizing the ranking objective function. Such metric-dependent setting limits LTR methods from two perspectives: (1) non-differentiable problem: directly optimizing ranking functions over a given ranking metric is inherently non-smooth, making the training process unstable and inefficient; (2) limited ranking utility: optimizing over one single metric makes it difficult to generalize well to other ranking metrics of interest. To address the above issues, we propose a novel listwise LTR framework for efficient and generalizable ranking purpose. Specifically, we propose a new differentiable ranking loss that combines a smooth approximation to the ranking operator with the average mean square loss per query. Then, we adapt gradient-boosting machines to minimize our proposed loss with respect to each list, a novel contribution. Finally, extensive experimental results confirm that our method outperforms the current state-of-the-art in information retrieval measures with similar efficiency.

Problem

• Learning-to-Rank
• ranking metric
• non-differentiable optimization
• generalization
• information retrieval

Innovation

• metric-agnostic
• differentiable ranking loss
• rank approximation
• gradient boosting
• listwise learning-to-rank