Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression

2026-06-10

AI Summary

This work addresses the challenge of KV cache inflation in multimodal large language models when processing long visual contexts, which leads to significant inference latency, while existing compression methods often discard critical visual information. To mitigate this, the authors propose BACON, a plug-and-play KV cache compression framework that introduces last-query attention as a complementary signal alongside windowed observation attention. The method further incorporates boundary-aware attention calibration, intra-layer consistency analysis, and cross-layer persistence modeling to accurately preserve essential visual tokens. Evaluated across diverse models, benchmarks, and compression budgets, BACON achieves an average performance gain of 7.5%, with improvements reaching up to 30.9% under extreme compression settings.

Abstract

Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning, but long visual contexts enlarge the KV cache and increase decoding latency. Existing compression methods rely on observation window attention for stable token-importance estimation, yet this aggregation can dilute sparse visual evidence and discard answer-critical tokens under aggressive compression. Therefore, we identify last-query attention as a complementary source for recovering such evidence, but its answer-irrelevant signals can mislead retention. We propose BACON, a plug-and-play method that calibrates observation window attention with last-query evidence and suppresses isolated noise via intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.

Problem

• Multimodal Large Language Models
• KV Cache Compression
• Visual Context
• Decoding Latency
• Token Importance

Innovation

• KV cache compression
• multimodal LLMs
• boundary attention
• attention calibration
• last-query attention