The Moving Eye: Enhancing VLA Spatial Generalization via Hybrid Dynamic Data Collection

2026-07-02

AI Summary

Existing Vision-Language-Action (VLA) models often suffer from spurious correlations due to their reliance on fixed camera viewpoints, resulting in poor spatial generalization. This work proposes a data-centric hybrid dynamic data collection strategy that systematically leverages a dual-arm coordination mechanism—one arm performing manipulation tasks while the other holds a mobile camera—to integrate continuous moving views with diverse static perspectives. The approach constructs three distinct data distribution modes: Fixed, Multi-Fixed, and Moving Views. By explicitly mitigating viewpoint bias through this design, the method substantially enhances the generalization performance of prominent VLA models—including ACT, Diffusion Policy, Pi0, and Gr00t—under novel camera poses and unseen object layouts.

Abstract

Vision-Language-Action (VLA) models have shown remarkable promise in generalized robotic manipulation. However, their spatial generalization remains fragile. We argue that simply increasing the number of viewpoints is insufficient. Models often fall into the trap of Shortcut Learning, latching onto spurious correlations (e.g., fixed relative poses between objects or between the camera and robot base) rather than learning true spatial relationships. In this work, we propose a data-centric solution to enhance VLA spatial generalization. We utilize a dual-arm setup where one arm performs manipulation while the other serves as a mobile environmental camera. We systematically evaluate three data distribution patterns: Fixed, Multi-Fixed, and Moving Views. Our findings reveal that a hybrid strategy, combining continuous camera motion with diverse static viewpoints, yields the best performance by substantially reducing spurious correlations while maintaining training stability. Our experiments demonstrate that this strategy mitigates spurious correlations, enabling VLAs to generalize to unseen camera poses and object configurations where simply adding more static viewpoints fails. Crucially, we reveal that the susceptibility to shortcut learning and the struggle with spatial generalization are universal characteristics shared across diverse architectures. Consequently, all evaluated models (ACT, Diffusion, and VLA models including Pi0 and Gr00t) benefit significantly from our mixed data strategy.

Problem

• spatial generalization
• Vision-Language-Action
• Shortcut Learning
• spurious correlations
• robotic manipulation

Innovation

• spatial generalization
• shortcut learning
• hybrid dynamic data collection
• Vision-Language-Action models
• mobile environmental camera