LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations

2026-05-08

AI Summary

Existing visual world models often suffer from cumulative errors and energy drift in long-horizon predictions due to insufficient physical constraints. This work proposes an implicit variational integrator with embedded physical structure that enforces the discrete principle of least action directly in the learned visual latent space. Rather than using the Lagrangian action functional merely as a regularizer, it serves as the core rule governing latent state transitions. By integrating implicit generalized coordinate encoding, a learnable discrete Lagrangian, and variational integration solvers, the method significantly enhances physical consistency, background stability, motion smoothness, and fidelity in both appearance and geometry over extended prediction horizons, as demonstrated in synthetic dynamics and robot interaction tasks.

Abstract

Learning predictive world models from visual observations is a core problem in embodied AI, with applications to model-based reinforcement learning and robotic planning. Existing latent world models typically generate future states with unconstrained neural transition functions, while modern video generation systems often prioritize perceptual plausibility or introduce physical structure through auxiliary losses, external guidance, or separate dynamics modules. As a result, long-horizon rollouts can remain weakly grounded in the physical principles that govern real dynamics, leading to compounding error, energy drift, and physically inconsistent futures. We propose Least Action World Models (LaWM), a latent world-modeling framework that operationalizes the Principle of Least Action in learned visual latent space: future rollouts are governed by a learned Lagrangian action functional rather than produced only by an unconstrained transition predictor. Our main technical realization is a latent variational integrator: LaWM encodes observations into learned generalized coordinates, learns a latent discrete Lagrangian over consecutive latent states, constructs a discrete action functional, and advances prediction by solving the corresponding discrete integration condition. Thus, physical structure is not merely used to score, regularize, or constrain a completed trajectory; it defines the latent transition rule itself. Because the transition is induced by a discrete variational principle, LaWM provides a structure-preserving bias for long-horizon visual prediction. Across physics-clean synthetic dynamics and embodied robot interaction benchmarks, LaWM improves physical invariance, background consistency, motion smoothness, and appearance and geometric prediction metrics over video-generation and world-model baselines.

Problem

• world models
• physical consistency
• long-horizon prediction
• visual observations
• embodied AI

Innovation

• Least Action Principle
• Latent World Models
• Variational Integrator
• Physical Consistency
• Lagrangian Dynamics