InforME: Improving Informativeness of Abstractive Text Summarization With Informative Attention Guided by Named Entity Salience

2025-10-07

AI Summary

To address the insufficient informativeness and weak capture of key content in abstractive text summarization, this paper proposes an information-aware attention mechanism grounded in named entity salience. The method tackles two core challenges: (1) modeling information alignment between source text and summary via optimal transport theory to enhance attention weights on salient entities; and (2) introducing a cumulative joint entropy reduction strategy over named entities to explicitly balance information density and salience. Integrating named entity recognition, attention mechanism reconstruction, and joint entropy minimization, the approach achieves a 1.3-point ROUGE-L improvement on CNN/Daily Mail and remains competitive on XSum. Human evaluation confirms significantly enhanced informativeness, reduced redundancy, and improved trade-off between conciseness and content completeness.

Technology Category

• Natural Language Processing: Summarization
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Machine Learning: Information Theory

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Abstractive text summarization is integral to the Big Data era, which demands advanced methods to turn voluminous and often long text data into concise but coherent and informative summaries for efficient human consumption. Despite significant progress, there is still room for improvement in various aspects. One such aspect is to improve informativeness. Hence, this paper proposes a novel learning approach consisting of two methods: an optimal transport-based informative attention method to improve learning focal information in reference summaries and an accumulative joint entropy reduction method on named entities to enhance informative salience. Experiment results show that our approach achieves better ROUGE scores compared to prior work on CNN/Daily Mail while having competitive results on XSum. Human evaluation of informativeness also demonstrates the better performance of our approach over a strong baseline. Further analysis gives insight into the plausible reasons underlying the evaluation results.

Problem

• Improving informativeness in abstractive text summarization models
• Enhancing named entity salience through informative attention mechanisms
• Optimizing summary quality using transport-based learning and entropy reduction

Innovation

• Optimal transport-based informative attention method
• Accumulative joint entropy reduction on named entities
• Improving informativeness with named entity salience guidance