OmniX: Any-view and Any-time 4D Reconstruction via Feed-forward Trajectory Fields

2026-07-12

AI Summary

Existing feedforward 4D reconstruction methods struggle to achieve complete dynamic scene reconstruction under large viewpoint variations, often neglecting foreground motion or being constrained to limited camera trajectories. This work proposes OmniX, a framework that, for the first time, enables full-scene 4D reconstruction at arbitrary viewpoints and time steps within a feedforward architecture. By decoupling static geometry from dynamic motion, OmniX introduces compact dynamic tokens to generate dense 3D trajectory fields, leveraging a low-rank sparse motion prior and a large-scale 4D dataset automatically synthesized using Unreal Engine 5. The method achieves state-of-the-art performance across multiple tasks, including dense 3D trajectory prediction, point tracking, video depth estimation, and camera pose estimation.

Abstract

Previous feed-forward 4D reconstruction methods either predict per-frame static point clouds, ignoring foreground motion, or estimate point cloud trajectories while being limited to small camera motions. This restricts their ability to aggregate observations over time and reconstruct complete dynamic scenes under large viewpoint changes. To address this limitation, we propose OmniX, a feed-forward 4D reconstruction framework that predicts dense 3D point trajectories for every pixel from videos with large camera motion. OmniX decouples dynamic motion modeling from static geometry prediction and represents motion using a compact set of dynamic tokens. By leveraging the sparse and low-rank structure of 3D motion, these tokens generate trajectory fields for all pixels across all images while efficiently preserving global interactions. To facilitate training, we further build an automatic UE5-based 4D data engine and introduce a large-scale dataset containing 80K scenes and 1.28M multi-view videos with full geometric annotations. OmniX achieves state-of-the-art performance on dense 3D point trajectory prediction and 3D point tracking, while also demonstrating competitive results on video depth estimation and camera pose estimation.

Problem

• 4D reconstruction
• dynamic scenes
• large camera motion
• point cloud trajectories
• viewpoint changes

Innovation

• 4D reconstruction
• trajectory fields
• dynamic tokens
• feed-forward framework
• large camera motion