PoLAR: Factorizing Extent and Mode in Latent Actions for Robot Policy Learning

2026-06-19

AI Summary

Existing latent action pretraining methods encode visual state transitions into unstructured, monolithic representations that conflate transition magnitude and pattern, thereby hindering downstream policy learning. This work proposes Polar Latent Actions (PoLAR), which explicitly disentangles magnitude and pattern in latent space for the first time: magnitude is encoded as radial distance, while directional components preserve transition patterns. The method leverages observed time intervals between states as a weak supervisory signal for magnitude and integrates radial geometric structure with hyperbolic space embeddings to enhance representational structure and transferability. Experiments demonstrate that PoLAR significantly outperforms current latent action baselines and strong pretrained vision-language-action (VLA) models on both simulated and real-world robotic tasks, consistently improving downstream policy performance.

Abstract

Latent action pretraining learns representations of visual change from pairs of observations, but existing methods typically encode each transition as a single unstructured representation that entangles transition extent and transition mode. We introduce Polar Latent Actions with Radial structure (PoLAR), which imposes a radial-direction structure on latent actions, encouraging radius to encode transition extent and direction to retain transition mode. PoLAR uses temporal offset between two observations as a weak proxy for transition extent, encouraging latent action from observation pairs separated by larger temporal gaps to occupy larger radii. We instantiate this structure in hyperbolic space, whose expanding volume with radius offers a natural fit for more diverse transition modes at larger extents. Across in-task and large-scale pretraining settings, PoLAR improves downstream policy performance in simulation and real-world robot experiments, outperforming latent action baselines and strong pretrained VLAs. These results suggest that the geometry of the latent action space is an important design choice for transferring visual pretraining to downstream robot policy learning.

Problem

• latent actions
• transition extent
• transition mode
• representation learning
• robot policy learning

Innovation

• latent actions
• radial structure
• hyperbolic space
• policy learning
• visual pretraining