Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models

2026-06-29

AI Summary

This work addresses the degraded performance of multimodal large language models in fine-grained visual reasoning, often caused by redundant image tokens diluting critical visual cues. To mitigate this issue without modifying the backbone architecture, the authors propose Lens—a lightweight framework that introduces learnable evidence tokens (LETs) to score the relevance of visual tokens conditioned on the input question. Low-relevance tokens are then adaptively perturbed with noise in the latent space, effectively performing visual denoising. The method adds only a single trainable control token and a compact noise generator, without increasing the inference sequence length. Experiments demonstrate consistent improvements across multiple visual question answering benchmarks, with gains of 2.4–6.4 points, and significant enhancements of 4.1–6.4 points on visual grounding tasks, substantially boosting multimodal reasoning capabilities.

Abstract

Multimodal large language models (MLLMs) often fail in fine-grained visual reasoning, as question-relevant visual cues are diluted by dense and redundant image tokens. Recent multimodal reasoning methods usually extend chain-of-thought from language models into visual or latent spaces, seeking to add intermediate reasoning states while overlooking the negative impact of redundant visual tokens. We propose LatEnt Noise maSk (Lens), a question-conditioned visual evidence purification framework that empowers MLLMs to reason with cleaner visual cues in latent space. Lens introduces a lightweight Lens Evidence Token (LET) to score which visual tokens support the current question and preserve them during decoding. Guided by the LET scores, it injects adaptive latent noise into low-relevance tokens, softly suppressing distractors without changing the model backbone or token sequence. With only one temporary learnable control token and a lightweight noise generator, Lens adds minimal overhead while improving the base MLLM by 2.4-6.4 points on most VQA datasets and by 4.1-6.4 points on grounding tasks. These results show that multimodal reasoning can benefit more directly from cleaner question-relevant visual evidence than from simply extending the reasoning trace.

Problem

• visual redundancy
• multimodal large language models
• fine-grained visual reasoning
• image tokens
• visual cues

Innovation

• Latent Noise Mask
• Visual Redundancy Reduction
• Multimodal Reasoning
• Evidence Token
• Adaptive Noise Injection