ATACompressor: Adaptive Task-Aware Compression for Efficient Long-Context Processing in LLMs

2025-12-06

SIGIR-AP

AI Summary

This work proposes ATACompressor, a task-aware dynamic context compression framework designed to address the challenge of long-context processing in large language models, where critical intermediate information is often lost and existing compression methods struggle to balance fidelity and efficiency. ATACompressor employs a selective encoder to identify task-relevant content and an adaptive controller that dynamically adjusts the compression ratio based on input length, enabling efficient, on-demand compression. Evaluated on benchmark datasets including HotpotQA, MSMARCO, and SQuAD, the method significantly outperforms current approaches, achieving higher compression efficiency while effectively preserving—and in some cases even enhancing—downstream task performance.

Technology Category

• Data Mining & Knowledge Management: Data Compression
• Natural Language Processing: Summarization
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Long-context inputs in large language models (LLMs) often suffer from the ''lost in the middle'' problem, where critical information becomes diluted or ignored due to excessive length. Context compression methods aim to address this by reducing input size, but existing approaches struggle with balancing information preservation and compression efficiency. We propose Adaptive Task-Aware Compressor (ATACompressor), which dynamically adjusts compression based on the specific requirements of the task. ATACompressor employs a selective encoder that compresses only the task-relevant portions of long contexts, ensuring that essential information is preserved while reducing unnecessary content. Its adaptive allocation controller perceives the length of relevant content and adjusts the compression rate accordingly, optimizing resource utilization. We evaluate ATACompressor on three QA datasets—HotpotQA, MSMARCO, and SQUAD—showing that it outperforms existing methods in terms of both compression efficiency and task performance. Our approach provides a scalable solution for long-context processing in LLMs. Furthermore, we perform a range of ablation studies and analysis experiments to gain deeper insights into the key components of ATACompressor. Our code is available at https://github.com/Cocobalt/ATACompressor.git.

Problem

• long-context processing
• lost in the middle
• context compression
• information preservation
• compression efficiency

Innovation

• Adaptive Compression
• Task-Aware
• Long-Context Processing
• Selective Encoding
• Large Language Models