MiCo: Multi-image Contrast for Reinforcement Visual Reasoning

2025-06-27

AI Summary

This work addresses two key challenges in multi-image fine-grained visual reasoning: reliance on human-annotated question-answer pairs and difficulty modeling cross-image logical relationships. Methodologically, we propose a self-supervised chain-of-reasoning framework that constructs image triplets to uncover intrinsic visual constraints, employs chain-of-thought prompting for stepwise inference, and integrates rule-guided reinforcement learning to encourage the model to autonomously attend to subtle visual differences and perform interpretable logical deduction—entirely without annotated QA pairs. Our primary contribution is the first integration of self-supervised contrastive learning with structured chain-of-reasoning, enabling fine-grained cross-image comparison and generalization to complex logical patterns. Experiments demonstrate significant improvements over existing unsupervised methods on multi-image reasoning benchmarks, while also exhibiting strong transferability to general vision tasks.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This work explores enabling Chain-of-Thought (CoT) reasoning to link visual cues across multiple images. A straightforward solution is to adapt rule-based reinforcement learning for Vision-Language Models (VLMs). However, such methods typically rely on manually curated question-answer pairs, which can be particularly challenging when dealing with fine grained visual details and complex logic across images. Inspired by self-supervised visual representation learning, we observe that images contain inherent constraints that can serve as supervision. Based on this insight, we construct image triplets comprising two augmented views of the same image and a third, similar but distinct image. During training, the model is prompted to generate a reasoning process to compare these images (i.e., determine same or different). Then we optimize the model with rule-based reinforcement learning. Due to the high visual similarity and the presence of augmentations, the model must attend to subtle visual changes and perform logical reasoning to succeed. Experiments show that, although trained solely on visual comparison tasks, the learned reasoning ability generalizes effectively to a wide range of questions. Without relying on any human-annotated question-answer pairs, our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks.

Problem

• Enable CoT reasoning across multiple visual cues
• Overcome reliance on manual QA pairs for VLMs
• Learn reasoning via self-supervised image comparisons

Innovation

• Self-supervised triplet image comparison training
• Rule-based reinforcement learning optimization
• Augmentation-driven visual reasoning generalization