SEER: Long-Context Reasoning via Selective Visual-Text Compression

2026-08-16

AI Summary

This study addresses the prohibitive inference costs of long-context large language models and the query-agnostic nature of existing compression methods by proposing SEER. This framework integrates selective visual scanning for key image filtering with on-demand text retrieval, enhanced by fine-tuning on tool-interaction trajectories to enable adaptive invocation. Consequently, SEER effectively balances compression efficiency with reasoning accuracy. Evaluated on the LongBench benchmark, it achieves an average accuracy of 51.11%, outperforming baselines such as Glyph-9B and Qwen3-8B. By innovatively combining selective vision-text compression with adaptive tool utilization, this work significantly improves token efficiency and overall performance in long-context tasks, offering a robust solution to computational constraints while maintaining high fidelity in complex reasoning scenarios.

Abstract

Long-context reasoning remains computationally expensive for large language models due to the quadratic complexity of attention over text tokens. Visual-text compression offers a promising alternative by rendering text into images and processing them with vision-language models, often reducing token usage. However, existing approaches apply uniform compression regardless of query relevance, potentially sacrificing precision where detailed extraction is required. We present SEER, a framework that learns to select query-relevant images through visual scanning and retrieve textual content only where needed, combining the efficiency of visual compression with the precision of text-based reasoning. Through supervised fine-tuning on tool-interaction trajectories, SEER learns adaptive tool invocation for selection and retrieval. Experiments on long-context benchmarks show that SEER improves extraction precision through selective text retrieval while retaining average prompt-token savings relative to full-text baselines. On LongBench, SEER achieves 51.11% average accuracy, outperforming the visual-text baseline Glyph-9B by 2.33 points and Qwen3-8B by 3.49 points. Code can be accessed at https://github.com/jiaweixu98/SEER

Problem

• Long-context reasoning
• Visual-text compression
• Computational efficiency
• Extraction precision
• Query relevance

Innovation

• Selective Visual-Text Compression
• Long-Context Reasoning
• Adaptive Tool Invocation
• Visual Scanning
• Token Efficiency