G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models

2026-06-23

AI Summary

This work addresses the limitation of existing vision-language-action (VLA) models that disregard known camera geometry in multi-camera setups, leading to visual representations misaligned with the true 3D space. To resolve this, the authors propose a camera-aware geometric module that injects calibrated geometric information into the visual token stream without altering the pre-trained VLA action space. The approach leverages intrinsic-conditioned ray embeddings, Projected Positional Encoding (PRoPE), and a bidirectional cross-view fusion mechanism. Notably, it requires neither depth sensors nor manual annotations, instead utilizing confidence-gated geometric supervision derived from a π³X teacher model. The method consistently improves performance across LIBERO, RoboCasa24, RoboTwin2.0, and real-world robotic platforms, with particularly pronounced gains on tasks sensitive to spatial reasoning and object relationships.

Abstract

Vision-language-action (VLA) models have made rapid progress in generalist robot manipulation by harnessing semantic knowledge from pretrained vision-language backbones, but their visual tokens remain grounded in 2D image coordinates rather than the calibrated geometry of the robot's cameras -- a mismatch especially pronounced in multi-camera setups, where views are coupled by known intrinsics and extrinsics yet processed as independent images. We propose G$^3$VLA, a camera-aware geometric module that injects calibrated structure into the visual-token stream of a pretrained VLA without altering its action space or imitation objective, combining intrinsic-conditioned ray embeddings, projective positional encoding (PRoPE), and bidirectional cross-view fusion. Geometric supervision is provided either from ground-truth point maps when available, or from confidence-gated $π^3$X teacher predictions, requiring no depth sensors or manual annotations. Instantiated on $π_0$, G$^3$VLA yields consistent gains across the LIBERO suites, RoboCasa24, RoboTwin2.0, and real-robot settings, with the largest improvements on spatially and object-sensitive tasks. We further validate on $π_{0.5}$ and GR00T 1.5, with results suggesting that geometric transfer is most effective when geometry-aware tokens have direct access to the action generation pathway. Our project page is at https://sites.google.com/view/g3vla

Problem

• Vision-Language-Action Models
• Geometric Inductive Bias
• Multi-camera Geometry
• Visual Token Representation
• Robot Manipulation

Innovation

• geometric inductive bias
• vision-language-action models
• camera-aware fusion
• projective positional encoding
• cross-view geometry