Think, Look, and Revise: Inconsistency-Aware Visual Self-Correction in MLLMs

2026-08-29

AI Summary

本文提出ReVISE框架，通过验证工具输出和动态错误恢复来解决多模态大语言模型中的感知瓶颈问题。

Abstract

Tool-augmented multimodal reasoning integrates external tools (e.g., object detection, depth estimation) into multimodal large language models (MLLMs) to address perceptual bottlenecks in complex visual tasks. However, existing approaches rarely verify tool outputs, limiting their ability to detect and recover from tool failures. We propose ReVISE, a framework that equips MLLMs with verification and dynamic error recovery for tool-augmented reasoning. ReVISE introduces (1) a curated training dataset that supervises reflective behaviors, enabling models to validate tool-derived evidence, reformulate queries when visual mismatches arise, and fall back to intrinsic grounding when external tools are unreliable; and (2) a reinforcement learning based targeted rewards that encourage internal reflection and penalize spatial misalignment. Experiments on several benchmarks demonstrate consistent improvements over existing methods, highlighting the importance of error detection and correction in tool-augmented multimodal reasoning.

Problem

• tool-augmented multimodal reasoning
• external tools
• error detection
• error recovery

Innovation

• Verification and Dynamic Error Recovery
• Reflective Behaviors Supervision
• Reinforcement Learning Based Targeted Rewards