Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization

2025-09-11

AI Summary

Multi-document summarization (MDS) faces dual challenges of fragmented information integration and weak thematic coherence. To address these, we propose a topic-guided generative framework synergizing explicit topic modeling with reinforcement learning (RL). Our approach makes three key contributions: (1) integrating explicit topic-label prompts to enhance cross-document thematic structure awareness; (2) designing a fine-grained reward function based on inter-sentence topic similarity to jointly optimize content selection and thematic alignment; and (3) incorporating this reward into the Group Relative Policy Optimization (GRPO) framework for stable, sample-efficient RL training. Evaluated on Multi-News and Multi-XScience, our method significantly outperforms strong baselines across ROUGE, BERTScore, and human evaluations—particularly in thematic relevance and informational completeness—demonstrating the effectiveness of topic-guided RL for MDS.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Generation
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Topic discovery and tracking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

A key challenge in Multi-Document Summarization (MDS) is effectively integrating information from multiple sources while maintaining coherence and topical relevance. While Large Language Models have shown impressive results in single-document summarization, their performance on MDS still leaves room for improvement. In this paper, we propose a topic-guided reinforcement learning approach to improve content selection in MDS. We first show that explicitly prompting models with topic labels enhances the informativeness of the generated summaries. Building on this insight, we propose a novel topic reward within the Group Relative Policy Optimization (GRPO) framework to measure topic alignment between the generated summary and source documents. Experimental results on the Multi-News and Multi-XScience datasets demonstrate that our method consistently outperforms strong baselines, highlighting the effectiveness of leveraging topical cues in MDS.

Problem

• Improving content selection in multi-document summarization
• Enhancing topic alignment between summaries and source documents
• Addressing coherence and relevance challenges in MDS

Innovation

• Topic-guided reinforcement learning for summarization
• Group Relative Policy Optimization framework integration
• Explicit topic prompting enhances summary informativeness