Flow-JEPA: Flow Matching for Robust Latent Dynamics in JEPA World Models

2026-08-28

AI Summary

本文提出Flow-JEPA，通过条件流匹配方法生成未来潜在状态序列，以解决JEPA世界模型中确定性自回归预测器易积累误差和对视觉干扰敏感的问题。

Abstract

Joint-Embedding Predictive Architectures (JEPAs) have shown strong potential for learning compact predictive representations, and LeWorldModel (LeWM) extends this paradigm to reconstruction-free latent world modeling from pixels. However, its deterministic autoregressive predictor generates future states through repeated one-step transitions, which can accumulate errors and remain sensitive to task-irrelevant visual perturbations. In this work, we propose Flow-JEPA (F-JEPA), a conditional flow matching dynamics model that jointly generates a sequence of future latent states conditioned on the current observation and actions. A Gaussian distribution serves as the flow source, exposing the vector field to perturbed latent trajectories as it learns to transport them toward clean future representations. This formulation retains the reconstruction-free JEPA framework while replacing point-wise transition regression with stochastic trajectory-level prediction. F-JEPA raises mean success from $86\%$ to $92\%$ under clean observations and from $67\%$ to $86\%$ under noisy conditions, suggesting that conditional flow matching provides a promising alternative to deterministic autoregressive dynamics in JEPA world models.

Problem

• JEPA
• autoregressive predictor
• latent states
• visual perturbations
• world models

Innovation

• conditional flow matching
• stochastic trajectory-level prediction
• Gaussian distribution