Multi-View Relational Distillation for Spatial Reasoning with Vision-Language Models

2026-08-11

AI Summary

Visual-language models exhibit limited performance on spatial reasoning tasks due to their fragile geometric representations, hindering their applicability in embodied intelligence and autonomous driving. This work proposes Multi-View Relation Distillation (MVRD), a novel approach that distills the structural relationships—specifically, cosine similarity patterns among image patches—observed by a teacher model across multiple viewpoints, rather than raw feature representations. By preserving the student model’s pre-trained vision-language alignment while substantially enhancing its geometric awareness, MVRD achieves significant performance gains. Experiments demonstrate that MVRD consistently outperforms both supervised fine-tuning and conventional feature distillation across multiple visual-language models, exhibiting superior generalization on 3D scene understanding tasks with lower computational overhead.

Abstract

Vision-language models (VLMs) have achieved strong image and video understanding, yet their visual-spatial representations remain geometrically fragile, leading to failures in spatial reasoning needed for embodied AI, robotics, and autonomous driving. Prior approaches to geometry grounding either fine-tune VLMs on spatial question answering, which can perpetuate spurious visual representations, or fuse features from large geometry-grounded vision models, which substantially increases model size at inference. Knowledge distillation from geometry-grounded vision models offers an alternative, but directly matching multi-view teacher features can disrupt the pretrained alignment between visual and textual representations, degrading object- and language-semantic capabilities. We propose multi-view relational distillation (MVRD), which distills patch-wise cosine similarities across views instead of the teacher features themselves. These relations encode geometric correspondences adequate for spatial understanding, while leaving the student representation underdetermined, allowing it to remain close to its pretrained vision- language space. Across representative VLMs, MVRD improves visual-spatial reasoning, outperforming supervised fine-tuning and feature distillation while approaching feature fusion methods with considerably fewer added parameters and lower latency. We show that MVRD makes visual representations more geometric while retaining language alignment, and generalizes to 3D scene understanding tasks such as object grounding, dense captioning, and question answering.

Problem

• spatial reasoning
• vision-language models
• geometric fragility
• visual-spatial representation
• embodied AI

Innovation

• multi-view relational distillation
• vision-language models
• spatial reasoning
• knowledge distillation
• geometric grounding