BUS: Brain-Inspired Unsupervised Self-Reflection for Advanced Multimodal Reasoning

2026-07-08

AI Summary

This work addresses the limited fine-grained and consistent reasoning capabilities of current vision-language models in complex tasks, as well as the reliance of existing self-reflection methods on extensive labeled data and their inability to enable explicit backtracking at test time. Inspired by the reverse prediction mechanism in neuroscience, this study introduces, for the first time, an unsupervised self-reflection framework into vision-language modeling. The proposed approach drives explicit backtracking and refinement of reasoning processes without requiring any labels and seamlessly integrates with both supervised fine-tuning and reinforcement learning. Evaluated across eight benchmarks, the method significantly enhances performance on complex visual reasoning tasks, surpassing strong baselines using only unlabeled data, thereby demonstrating the critical role of reverse prediction in multimodal reasoning.

Abstract

Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods aim to train models to facilitate self-reflective reasoning, i.e., reviewing and improving the generated reasoning. However, they require large volumes of annotated data and lack explicit reflective behavior during test time. This work aims to bridge this gap through inspiration from neuroscience. The human brain exhibits efficient backward prediction, i.e., predicting which current states are likely to precede a given future state. In this work, we first verify that mainstream VLMs can perform backward prediction, similar to the human brain. Then, we propose Brain-inspired Unsupervised Self-reflection (BUS), a label-free training framework to enhance reflective reasoning capability in challenging image analysis. BUS enables VLMs to perform backward prediction and provide explicit learning signals on data without ground-truth labels. In this way, BUS eliminates reliance on annotated data while improving reasoning performance. Notably, BUS is compatible with popular fine-tuning methods, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Finally, extensive experiments on 8 benchmarks demonstrate the effectiveness of BUS across a wide range of complex visual tasks. It achieves notable improvements over the base model while using only unlabeled training data. Our experimental findings validate that backward prediction capability is critical for VLM reasoning.

Problem

• Vision-Language Models
• Self-Reflective Reasoning
• Backward Prediction
• Unsupervised Learning
• Multimodal Reasoning

Innovation

• backward prediction
• unsupervised self-reflection
• vision-language models
• brain-inspired learning
• multimodal reasoning