(How) Do MLLMs Report Bistable Images Like Humans?

2026-09-05

AI Summary

研究使用LLaVA模型探讨多模态大语言模型如何像人类一样报告双稳态图像，通过视觉和语言操控测试其内部计算机制。

Abstract

Bistable images such as the duck-rabbit are classic stimuli in which one image supports multiple mutually incompatible interpretations, typically reported one at a time in humans. We ask whether multimodal large language models (MLLMs) show similar report behavior and what internal computations support it. Using the LLaVA family, we study two tractable dimensions: modulability, whether reports can be biased by bottom-up visual cues and top-down linguistic priors, and exclusivity, whether responses commit to a single interpretation. We test both on the canonical duck-rabbit and on synthetic Visual Anagrams to mitigate memorization confounds. Behaviorally, both visual and linguistic manipulations systematically shift reports in human-consistent ways, while responses remain predominantly exclusive. Mechanistically, these effects arise from competing image-token representations, distinct pathways for bottom-up and top-down modulation, and a link between exclusive reporting and object-count encoding. Code and data are available at https://github.com/rtakatsky/mllm-bistable-images.

Problem

• bistable images
• multimodal large language models
• report behavior
• internal computations

Innovation

• multimodal large language models
• bistable images
• modulability
• exclusivity
• object-count encoding