ViSRA: A Video-based Spatial Reasoning Agent for Multi-modal Large Language Models

2026-05-11

AI Summary

Existing multimodal large language models struggle to effectively leverage explicit spatial information for 3D spatial reasoning during inference and often rely heavily on extensive human-annotated data and post-training. This work proposes ViSRA, a training-free, video-driven spatial reasoning agent that enhances the spatial understanding capabilities of multimodal large language models by modularly and plug-and-play integrating video inputs with explicit 3D cues provided by expert spatial perception models. ViSRA establishes the first human-aligned 3D spatial reasoning framework that requires neither post-training nor manual annotations. It achieves substantial performance gains across multiple benchmarks and unseen tasks, with absolute accuracy improvements of up to 15.6% and 28.9% over baseline methods, demonstrating remarkable generalization and scalability.

Abstract

Recent advances in Multi-modal Large Language Models (MLLMs) target 3D spatial intelligence, yet the progress has been largely driven by post-training on curated benchmarks, leaving the inference-time approach relatively underexplored. In this paper, we take a training-free perspective and introduce ViSRA, a human-aligned Video-based Spatial Reasoning Agent, as a framework to probe the spatial reasoning mechanism of MLLMs. ViSRA elicits spatial reasoning in a modular and extensible manner by leveraging explicit spatial information from expert models, enabling a plug-and-play flexible paradigm. ViSRA offers two key advantages: (1) human-aligned and transferable 3D understanding rather than task-specific overfitting; and (2) no post-training computational cost along with heavy manual curation of spatial reasoning datasets. Experimental results demonstrate consistent improvement across a set of MLLMs on both existing benchmarks and unseen 3D spatial reasoning tasks, with ViSRA outperforming baselines by up to a 15.6% and 28.9% absolute margin respectively.

Problem

• spatial reasoning
• Multi-modal Large Language Models
• 3D spatial intelligence
• inference-time approach
• video-based reasoning

Innovation

• training-free
• spatial reasoning
• video-based agent
• multi-modal LLMs
• 3D understanding