Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs

2026-08-09

AI Summary

This work addresses the high computational overhead of prefilling and key-value (KV) caching in Omni-modal large language models when processing long audio-visual sequences, a challenge exacerbated by existing compression methods that overlook modality-specific characteristics across LLM boundaries. The authors propose A-PACK, a two-stage framework: first, audio pruning is deferred until after multimodal interaction, leveraging local audio-visual dynamics—rather than per-token alignment—to guide video compression; second, within the LLM, query-conditioned analysis of multimodal interactions enables progressive pruning of low-relevance audio-visual tokens and their associated KV caches. Key findings reveal that audio tokens exhibit higher information density and representational diversity than video tokens, and this study presents the first approach to jointly prune cross-modal KV caches inside the LLM. Evaluated on Qwen2.5-Omni-7B/3B, the method reduces prefill FLOPs by up to 78% and improves decoding throughput by up to 2.21×.

Abstract

Omni-modal LLMs jointly process audio, video, and text, but long multimodal sequences incur substantial prefill and KV-cache costs. Existing omni-modal compression methods primarily focus on pre-LLM token reduction, leaving modality-specific compression across the LLM boundary underexplored. We propose A-PACK, a two-stage framework that defers audio pruning until query-conditioned multimodal interactions emerge. Our analysis shows that audio exhibits higher task-relevant information density and representational diversity per token than video. We further find that local audio-visual dynamics provide a more effective cue for visual selection than token-wise matching. We therefore preserve audio and compress video with local dynamics before the LLM, then progressively prune low-relevance audio and visual tokens and their KV-cache entries inside the LLM. Across four benchmarks on Qwen2.5-Omni-7B/3B, A-PACK achieves the strongest average performance among the evaluated prior methods while reducing prefill FLOPs by up to 78% and improving decoding throughput by up to 2.21x.

Problem

• Omni-LLMs
• audio pruning
• multimodal compression
• KV-cache
• prefill cost

Innovation

• deferred audio pruning
• local audio-visual dynamics
• omni-modal LLMs
• KV-cache compression
• multimodal token reduction