A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods

2024-03-05

arXiv.org

AI Summary

Existing ATS surveys predominantly adopt theoretical taxonomies and fail to address the paradigm shift induced by large language models (LLMs). This paper proposes a “process-oriented” classification framework that systematically maps the end-to-end ATS pipeline—from preprocessing and content selection to generation and evaluation—with particular emphasis on LLM-driven summarization advances over the past two years, including prompt engineering, fine-tuning, and retrieval-augmented strategies based on models such as GPT and LLaMA. Our contributions are threefold: (1) the first practical, deployment-oriented ATS process taxonomy; (2) the first comprehensive survey dedicated to LLM-based summarization; and (3) a unified analytical perspective integrating standard metrics (e.g., ROUGE) and neural evaluation methods (e.g., BERTScore) to characterize technical trajectories in the LLM era. The work serves as an authoritative reference for both industrial implementation and academic research.

Technology Category

• Natural Language Processing: Summarization
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Data Visualization & Summarization

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Automatic Text Summarization (ATS), utilizing Natural Language Processing (NLP) algorithms, aims to create concise and accurate summaries, thereby significantly reducing the human effort required in processing large volumes of text. ATS has drawn considerable interest in both academic and industrial circles. Many studies have been conducted in the past to survey ATS methods; however, they generally lack practicality for real-world implementations, as they often categorize previous methods from a theoretical standpoint. Moreover, the advent of Large Language Models (LLMs) has altered conventional ATS methods. In this survey, we aim to 1) provide a comprehensive overview of ATS from a ``Process-Oriented Schema'' perspective, which is best aligned with real-world implementations; 2) comprehensively review the latest LLM-based ATS works; and 3) deliver an up-to-date survey of ATS, bridging the two-year gap in the literature. To the best of our knowledge, this is the first survey to specifically investigate LLM-based ATS methods.

Problem

• Provide a process-oriented overview of ATS for real-world applications.
• Review latest LLM-based methods in automatic text summarization.
• Bridge the two-year gap in ATS literature with updated survey.

Innovation

• Process-Oriented Schema for practical ATS implementation
• Comprehensive review of LLM-based ATS methods
• Bridging two-year gap in ATS literature