Large Vision Models Can Solve Mental Rotation Problems

2025-09-18

AI Summary

This study investigates whether modern vision transformers—specifically ViT, CLIP, DINOv2, and DINOv3—exhibit human-like spatial reasoning capabilities in mental rotation tasks, probing their implicit geometric understanding. Method: We employ hierarchical representation probing across three stimulus modalities—block figures, textual stimuli, and natural object images—under systematically varied complexity conditions (rotation angle, occlusion level). Contribution/Results: Self-supervised models—especially DINOv2 and DINOv3—significantly outperform supervised counterparts; discriminative representations peak at intermediate network layers; and performance degrades monotonically with increasing rotation angle and occlusion, mirroring human cognitive constraints. This work provides the first empirical evidence that large vision models implicitly encode mental rotation mechanisms, demonstrating that self-supervised pretraining fosters geometric structure learning. These findings establish critical empirical grounding for modeling spatial cognition in AI systems.

Technology Category

• Computer Vision: Large Vision Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Representation Learning

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Mental rotation is a key test of spatial reasoning in humans and has been central to understanding how perception supports cognition. Despite the success of modern vision transformers, it is still unclear how well these models develop similar abilities. In this work, we present a systematic evaluation of ViT, CLIP, DINOv2, and DINOv3 across a range of mental-rotation tasks, from simple block structures similar to those used by Shepard and Metzler to study human cognition, to more complex block figures, three types of text, and photo-realistic objects. By probing model representations layer by layer, we examine where and how these networks succeed. We find that i) self-supervised ViTs capture geometric structure better than supervised ViTs; ii) intermediate layers perform better than final layers; iii) task difficulty increases with rotation complexity and occlusion, mirroring human reaction times and suggesting similar constraints in embedding space representations.

Problem

• Evaluating vision models on mental rotation tasks
• Comparing self-supervised and supervised ViTs performance
• Analyzing layer-wise geometric representation capabilities

Innovation

• Self-supervised ViTs capture geometric structure better
• Intermediate layers outperform final layers
• Task difficulty mirrors human reaction times