Learning from Self Critique and Refinement for Faithful LLM Summarization

2025-12-04

AI Summary

Large language models (LLMs) suffer from hallucination in long-document summarization, and existing self-supervised methods either rely on external teacher models or entail costly test-time iteration. Method: We propose a self-supervised training framework that requires no auxiliary models or test-time computation. Its core innovation is the first unified integration—within a single model—of self-critique, iterative refinement, and preference learning: the model generates high-quality critique signals to construct preference data, which then drives preference optimization. Contribution/Results: Experiments on XSUM, CNN/DM, and SAMSum demonstrate that our method consistently outperforms state-of-the-art self-supervised baselines in faithfulness while maintaining or improving overall summary quality. The approach is both computationally efficient and practically deployable.

Technology Category

• Natural Language Processing: Summarization
• Machine Learning: Unsupervised & Self-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) often suffer from hallucinations: output content that is not grounded in the input context, when performing long-form text generation tasks such as summarization. Prior works have shown that hallucinations can be reduced by iteratively critiquing and refining previously generated outputs using either the same model or a more powerful teacher model as the critique. However, these approaches either require additional test-time compute or assume access to more powerful teacher models, making them costly and less practical. In this work, we propose Self Critique and Refinement-based Preference Optimization (SCRPO), which is a self-supervised training framework that first constructs a preference dataset by leveraging the LLM's own critique and refinement capabilities, and then applies preference learning to improve the same LLM for faithful summarization. Experiments on three summarization benchmarks (XSUM CNNDM and SAMSum), demonstrate that our approach outperforms state-of-the-art self-supervised learning methods in terms of faithfulness metrics while either maintaining or improving other metrics that measure the overall quality of the summary. Moreover, compared to test-time refinement, our approach not only improves efficiency but also results in more faithful summaries.

Problem

• Reduces hallucinations in LLM summarization tasks
• Uses self-supervised training to avoid costly test-time refinement
• Improves faithfulness without requiring more powerful teacher models

Innovation

• Self-supervised training framework using self critique
• Constructs preference dataset from model's own refinement
• Applies preference learning to enhance summarization faithfulness