TIAO: Token Importance-Aware Policy Optimization for Text Summarization

2026-09-15

AI Summary

本文针对文本摘要任务中忽视单个词汇重要性的问题，提出了一种基于强化学习的、能识别关键词汇并调整轨迹优势的新策略TIAO。

Abstract

Text summarization requires models to condense content while preserving key qualities such as consistency and coherence. Large language models (LLMs) have shown strong performance on this task and can be further improved through reinforcement learning (RL). However, most existing methods apply reward signals directly to undifferentiated token sequences, overlooking the varying importance of individual tokens to word and sentence level quality in summarization. In this paper, we propose Token Importance-Aware Policy Optimization (TIAO), a novel reinforcement learning strategy that explicitly leverages token-importance awareness. Specifically, TIAO identifies core tokens based on token dependency and reweights a trajectory's advantage according to its overall dependencies. Experiments on the real world dataset show that our TIAO achieves highly competitive results, and that a 7B foundation model enhanced by TIAO performs comparably to GPT-4 and GPT-5-nano. Code is available at https://github.com/TechCloud-x/TIAO

Problem

• Text Summarization
• Token Importance
• Reinforcement Learning

Innovation

• Token Importance
• Policy Optimization
• Reinforcement Learning
• Text Summarization