Generating Query-Relevant Document Summaries via Reinforcement Learning

2025-08-11

AI Summary

E-commerce search systems often rely solely on product titles for ranking due to strict latency constraints, leading to insufficient intent matching; while full product descriptions contain richer semantic information, their length and computational cost hinder real-time integration with high-fidelity rerankers such as cross-encoders. To address this, we propose ReLSum—a reinforcement learning-based framework that end-to-end trains a large language model (LLM) to generate query-aware, semantically aligned, and highly concise product description summaries. The LLM is optimized using relevance scores from a cross-encoder as reward signals. Crucially, ReLSum is the first approach to explicitly align the summarization objective with the downstream ranking objective, eliminating the objective mismatch inherent in conventional two-stage pipelines. Experiments demonstrate significant improvements in offline recall and NDCG, alongside measurable gains in online user engagement, validating its effectiveness and scalability in large-scale industrial settings.

Technology Category

• Natural Language Processing: Summarization
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

E-commerce search engines often rely solely on product titles as input for ranking models with latency constraints. However, this approach can result in suboptimal relevance predictions, as product titles often lack sufficient detail to capture query intent. While product descriptions provide richer information, their verbosity and length make them unsuitable for real-time ranking, particularly for computationally expensive architectures like cross-encoder ranking models. To address this challenge, we propose ReLSum, a novel reinforcement learning framework designed to generate concise, query-relevant summaries of product descriptions optimized for search relevance. ReLSum leverages relevance scores as rewards to align the objectives of summarization and ranking, effectively overcoming limitations of prior methods, such as misaligned learning targets. The framework employs a trainable large language model (LLM) to produce summaries, which are then used as input for a cross-encoder ranking model. Experimental results demonstrate significant improvements in offline metrics, including recall and NDCG, as well as online user engagement metrics. ReLSum provides a scalable and efficient solution for enhancing search relevance in large-scale e-commerce systems.

Problem

• Generating concise query-relevant product summaries for search ranking
• Overcoming limitations of product titles lacking query intent details
• Aligning summarization and ranking objectives via reinforcement learning

Innovation

• Reinforcement learning for query-relevant summaries
• LLM-generated summaries for cross-encoder ranking
• Relevance scores as rewards for alignment