DynTrace: Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs

2026-07-14

AI Summary

This work addresses the challenge that multimodal large language models struggle to disentangle genuine object motion from apparent motion induced by camera movement in continuous dynamic scenes. To this end, the authors propose DynTrace, a novel framework that introduces, for the first time, a training-free mechanism for tracking evidence of dynamic objects. By leveraging geometric reprojection, DynTrace constructs Dynamic Trajectory Visualizations (DTV) in world coordinates and incorporates Dynamic Trace Tokens (DT-Tokens) and Dynamic Trace Graphs (DTGs) to inject geometrically grounded visual priors and structured spatiotemporal traces into the model. This approach effectively decouples true object dynamics from viewpoint changes, significantly enhancing the 4D spatiotemporal reasoning capabilities of open-source multimodal large language models on benchmarks including Dyn-Bench, VLM4D, and DSI-Bench.

Abstract

4D spatio-temporal reasoning, jointly modeling 3D spatial structure and temporal evolution, is essential for understanding dynamic worlds and enabling embodied interaction. While current Multimodal Large Language Models (MLLMs) show strong capabilities in static scene understanding and coarse-grained 4D tasks, they still have notable limitations in continuous dynamic scene perception, especially in tracking dynamic object evidence for coherent 4D spatio-temporal reasoning. This shortcoming stems mainly from relying on sparse frame-level observations, fragmenting continuous dynamic cues and leaving models unable to disentangle genuine object dynamics from camera-induced apparent motion. Inspired by humans tracking dynamic cues while compensating for viewpoint changes, we propose DynTrace, a training-free framework for 4D spatio-temporal reasoning with two complementary components. Dynamic Trajectory Visualization (DTV) reprojects world-coordinate trajectories onto the image plane, providing geometry-informed visual priors that disentangle genuine object dynamics from camera-induced apparent motion. Meanwhile, the Dynamic Trace Token (DT-Token), organized into a Dynamic Trace Graph (DTG), tracks object-level dynamic cues, trace evolution, and key moments, maintaining continuous dynamic object evidence for coherent 4D reasoning. Together, these two components equip MLLMs with continuously tracked dynamic object evidence, grounded in geometry-informed visual priors and structured spatio-temporal traces. DynTrace consistently improves open-source MLLMs, achieving state-of-the-art results on Dyn-Bench, VLM4D, and DSI-Bench, validating the importance of tracking dynamic object evidence for robust 4D spatio-temporal reasoning.

Problem

• 4D spatio-temporal reasoning
• dynamic object tracking
• multimodal large language models
• camera-induced motion
• continuous dynamic perception

Innovation

• 4D spatio-temporal reasoning
• Dynamic Trajectory Visualization
• Dynamic Trace Token
• geometry-informed visual priors
• Multimodal Large Language Models