A Decoding Algorithm for Length-Control Summarization Based on Directed Acyclic Transformers

2025-02-06

AI Summary

This work addresses the problem of controllable abstractive summarization under strict length constraints. We propose a novel decoding framework based on the Directed Acyclic Transformer (DAT), the first application of DAT to summarization, which explicitly models multi-segment path connections. Our SeqMAP decoding algorithm performs exact marginalization over valid paths to achieve maximum a posteriori (MAP) summarization while guaranteeing hard length compliance. To jointly optimize summary quality and constraint adherence, we integrate length-pruned beam search with a dedicated summary reranker. Evaluated on Gigaword and DUC2004, our method achieves state-of-the-art ROUGE scores—while strictly satisfying character- or token-level budget constraints—establishing a new benchmark for length-controlled summarization.

Technology Category

• Natural Language Processing: Summarization
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Length-control summarization aims to condense long texts into a short one within a certain length limit. Previous approaches often use autoregressive (AR) models and treat the length requirement as a soft constraint, which may not always be satisfied. In this study, we propose a novel length-control decoding algorithm based on the Directed Acyclic Transformer (DAT). Our approach allows for multiple plausible sequence fragments and predicts a emph{path} to connect them. In addition, we propose a Sequence Maximum a Posteriori (SeqMAP) decoding algorithm that marginalizes different possible paths and finds the most probable summary satisfying the length budget. Our algorithm is based on beam search, which further facilitates a reranker for performance improvement. Experimental results on the Gigaword and DUC2004 datasets demonstrate our state-of-the-art performance for length-control summarization.

Problem

• Develops length-control summarization algorithm
• Uses Directed Acyclic Transformer for decoding
• Ensures summaries meet specific length requirements

Innovation

• Directed Acyclic Transformer decoding
• Sequence Maximum a Posteriori algorithm
• Beam search with reranker enhancement