OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models

2026-07-03

AI Summary

This work addresses the high computational cost of multimodal large language models when processing audio-visual inputs, which stems from generating an excessive number of tokens. Existing token compression methods often overlook the temporal locality of query-relevant evidence and assume uniform information density across modalities. To overcome these limitations, the authors propose a training-free, query-guided token compression approach that enables the first modality-independent yet symmetric importance evaluation for audio and video tokens. Specifically, query-guided attention scores are used to estimate token importance per modality, followed by a modality-balanced compression strategy that preserves critical information while maintaining cross-modal alignment. Evaluated at a 25% token retention rate, the method significantly outperforms existing approaches on four benchmarks, including DailyOmni, achieving 59.40% accuracy with Qwen2.5-Omni-7B and accelerating prefilling by 1.38×.

Abstract

Omni modal large language models (OmniLLMs) have attracted wide attention for their ability to jointly process audio and video, but they generate large token sequences under audio-visual inputs, leading to substantial inference cost. Existing audio-visual token compression methods often rely on unimodal guidance, overlooking the temporal locality of query-relevant evidence in audio-visual inputs and implicitly assuming that the two modalities share a temporally aligned information density distribution. We propose \textbf{OmniFocus}, a training-free query-guided token compression method for OmniLLMs that performs independent importance estimation for video and audio, enabling a modality-symmetric compression design that preserves modality-specific salient evidence while maintaining audio-visual alignment, thereby mitigating the modality bias issue that can arise from unimodal-guided compression. Experiments on the Qwen2.5-Omni model family across four audio-visual benchmarks show that OmniFocus maintains strong compressed performance at low token retention ratios and outperforms existing baselines on several major benchmark scores at 25\% token retention. On DailyOmni with Qwen2.5-Omni-7B at 25\% token retention, OmniFocus maintains 59.40 accuracy while delivering up to 1.38$\times$ prefill speedup relative to the full-token baseline, highlighting a favorable practical accuracy-efficiency trade-off.

Problem

• token compression
• omni-modal LLMs
• modality bias
• audio-visual alignment
• query-guided

Innovation

• token compression
• query-guided
• modality-balanced
• OmniLLM
• training-free