Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos

2026-06-23

AI Summary

This work addresses the scarcity of real-world teleoperation data that hinders the development of Vision-Language-Action (VLA) models. Existing approaches often misapply pseudo-action supervision derived from synthetic videos, failing to account for the asymmetric preservation of geometric and control information during synthesis. To resolve this, the authors propose the Geometry-guided Representation Alignment (GRA) framework, which explicitly decouples these two aspects: only reliable spatial geometric trajectories—such as 2D end-effector waypoints—from synthetic human videos supervise the visual perception module, while action prediction is trained exclusively on real demonstrations. Through a spatial representation anchoring mechanism and an auxiliary 2D path loss, GRA enables consistent geometric transfer across domains. Experiments demonstrate that this approach significantly outperforms pseudo-action baselines and substantially narrows the performance gap with fully real-data-trained policies under limited real-world supervision.

Abstract

Vision-Language-Action (VLA) models require large-scale video-action pairs, yet real teleoperation remains scarce. While generated robot videos offer a scalable alternative, existing methods treat them as real robot data by recovering pseudo-actions from synthesized pixels. We argue that deriving low-level control from generated visuals is a mismatched abstraction. A video captures only \emph{geometry}: the spatial trajectory representing the \emph{where} of a task. A real demonstration captures \emph{control}: the exact motor commands representing the \emph{how}. Human-to-robot video generation preserves these unequally: the visible geometry survives the generation process, while the underlying control signals are lost. This \textbf{Asymmetric Preservation Principle} dictates a clean rule: this surviving geometry should solely supervise visual perception, leaving control to real demonstrations. Following this principle, we propose \textbf{GRA} (\textbf{G}eometry-guided \textbf{R}epresentation \textbf{A}lignment), which extracts the geometric content as future 2D end-effector waypoints, computed from the source human video through pose estimation, retargeting, simulation, and calibrated projection, and routes them to the VLA vision backbone via an auxiliary 2D head. The action head is trained on real demonstrations only. During fine-tuning, the waypoint loss persists as a \textbf{spatial representation anchor} that prevents the backbone from losing its geometric grounding. On real-robot tasks, GRA outperforms pseudo-action baselines under matched data budgets and narrows the gap to policies trained with substantially more real demonstrations, suggesting that correctly routed geometry bridges generated videos to robot policies more reliably than recovered actions.

Problem

• Vision-Language-Action
• synthetic robot videos
• geometry preservation
• action supervision
• asymmetric preservation

Innovation

• Geometry-Guided Learning
• Vision-Language-Action Models
• Asymmetric Preservation Principle
• Synthetic Robot Videos
• Spatial Representation Anchor