Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots

2026-08-10

AI Summary

Existing multimodal large language models lack fine-grained, unsupervised visual perception learning mechanisms during self-improvement. This work proposes the CVPD framework, which for the first time enables a fully self-contained, online, token-level visual self-distillation process. By contrasting model responses to full images and their local crops, CVPD employs a three-gate counterfactual criterion to automatically identify visual blind spots and generate dense contrastive supervision signals—without requiring external annotations, auxiliary tools, or stronger teacher models. Evaluated on Qwen3-VL-8B-Instruct, CVPD significantly outperforms six self-evolution baselines, achieving gains of 3.60, 3.38, and 3.08 points on OCRBench and MMStar benchmarks in fine-grained perception and logical reasoning tasks, respectively, with no observed performance degradation.

Abstract

Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce \textbf{CVPD} (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning. We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression. It achieves gains of $+3.60$ on OCRBench, $+3.38$ on MMStar Fine-Grained Perception, and $+3.08$ on MMStar Logical Reasoning, while maintaining or improving performance on broader multimodal benchmarks.

Problem

• multimodal large language models
• visual self-distillation
• self-contained learning
• dense supervision
• visual blind spots

Innovation

• self-distillation
• counterfactual reasoning
• visual blind spots
• multimodal LLMs
• dense supervision