Teaching Large Language Models Number-Focused Headline Generation With Key Element Rationales

2025-02-05

AI Summary

Digital-sensitive news headline generation requires balancing textual quality with numerical accuracy, yet existing approaches typically prioritize one aspect at the expense of the other. To address this, we propose a Topic-Entity-Number (TEN)-driven chain-of-thought (CoT) pedagogical framework. Our method introduces the first rational TEN annotation paradigm and establishes a teacher–student large language model (LLM) co-teaching mechanism. Through CoT-guided instruction, rational supervision signal distillation, and joint fine-tuning, it simultaneously enhances numerical reasoning capability and topic alignment fidelity. Extensive experiments across multiple benchmarks demonstrate that our approach significantly outperforms state-of-the-art methods on both textual quality metrics (e.g., BLEU, ROUGE) and numerical accuracy. Notably, it achieves, for the first time, synergistic improvement in mathematical rigor and linguistic expressiveness—establishing a novel paradigm for rationality-driven, controllable text generation.

Technology Category

• Natural Language Processing: Generation
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Number-focused headline generation is a summarization task requiring both high textual quality and precise numerical accuracy, which poses a unique challenge for Large Language Models (LLMs). Existing studies in the literature focus only on either textual quality or numerical reasoning and thus are inadequate to address this challenge. In this paper, we propose a novel chain-of-thought framework for using rationales comprising key elements of the Topic, Entities, and Numerical reasoning (TEN) in news articles to enhance the capability for LLMs to generate topic-aligned high-quality texts with precise numerical accuracy. Specifically, a teacher LLM is employed to generate TEN rationales as supervision data, which are then used to teach and fine-tune a student LLM. Our approach teaches the student LLM automatic generation of rationales with enhanced capability for numerical reasoning and topic-aligned numerical headline generation. Experiments show that our approach achieves superior performance in both textual quality and numerical accuracy.

Problem

• Enhance LLMs for numerical headline generation.
• Improve textual quality and numerical accuracy.
• Use TEN rationales for better topic alignment.

Innovation

• Chain-of-thought framework
• TEN rationales supervision
• Teacher-student LLM fine-tuning