Marrying Optimal Transport and ODEs for Unified Continuous-Time 4D Reconstruction and Tracking

2026-08-10

AI Summary

Existing unified approaches for 4D reconstruction and point tracking are limited by discrete-time prediction or heuristic interpolation, often failing to preserve kinematic consistency. This work proposes a continuous velocity field modeling framework grounded in optimal transport (OT) and ordinary differential equations (ODEs), enabling velocity prediction at arbitrary time instants through a Flow Matching Guided Decoder. We introduce an integral consistency training strategy that facilitates end-to-end optimization without requiring ground-truth velocity supervision. Furthermore, we establish the first kinematics-aware evaluation benchmark tailored for continuous-time settings. The proposed method achieves state-of-the-art performance on both 4D reconstruction and point tracking tasks and demonstrates substantial gains on the newly introduced benchmark.

Abstract

Existing unified 4D reconstruction and point tracking approaches typically rely on heuristic interpolations or just predict at integer timestamps, lacking kinematic coherence and failing to model dynamics at any arbitrary timestamp. In this paper, we propose Uni4R, a framework that unifies these tasks by learning continuous velocity fields through the synergy of Optimal Transport (OT) and Ordinary Differential Equation (ODE). Importantly, this continuous velocity field acts as a kinematic prior that mutually benefits both 4D reconstruction and point tracking. Specifically, we propose the Flow Matching Guided Decoder (FMGD). A global velocity branch first extracts anchor features that capture the global dynamic state of the sequence. Then, FMGD leverages Flow Matching (FM) theory to formulate a probability path defined by OT on the anchor feature manifold, instantiating it as FM-guided velocity features for velocity prediction. This establishes a robust kinematic inductive bias. Meanwhile, a point reconstruction branch provides geometric features. The local velocity prediction module then joint above features and time embeddings, to decode velocities at arbitrary timestamps. To overcome the absence of high-quality ground-truth velocities in fractional frames, we propose an integral-consistency training strategy. This strategy uses an ODE solver to integrate velocities to recover target pointmaps, enabling the model to be supervised end-to-end directly from integer timestamps. Experimental results demonstrate that Uni4R achieves SOTA performance in both 4D reconstruction and point tracking, and achieves SOTA in our new kinematics-aware benchmark at continuous time.

Problem

• 4D reconstruction
• point tracking
• kinematic coherence
• continuous-time modeling
• dynamic modeling

Innovation

• Optimal Transport
• Ordinary Differential Equation
• Flow Matching
• Continuous-Time 4D Reconstruction
• Kinematic Prior