SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation

2026-04-04

AI Summary

Existing video generation methods struggle to simultaneously control camera motion and object dynamics accurately, often conflating parallax with true motion due to reliance on ambiguous 2D cues. This work proposes a unified motion control framework that enables stable viewpoint transitions through explicit camera trajectory modeling and geometry-aware representation. By integrating 2D guidance with 3D trajectory embeddings, the approach ensures spatially consistent object dynamics. It achieves, for the first time, joint control over both camera and object motion, and introduces RealCOD-25K—a large-scale dataset containing real camera poses and object 3D trajectories—to address the critical lack of annotated data in this domain. Experiments demonstrate significant improvements over existing methods in visual fidelity, camera controllability, and object motion accuracy. The code and dataset are publicly released.

Abstract

Controlling both camera motion and object dynamics is essential for coherent and expressive video generation, yet current methods typically handle only one motion type or rely on ambiguous 2D cues that entangle camera-induced parallax with true object movement. We present SymphoMotion, a unified motion-control framework that jointly governs camera trajectories and object dynamics within a single model. SymphoMotion features a Camera Trajectory Control mechanism that integrates explicit camera paths with geometry-aware cues to ensure stable, structurally consistent viewpoint transitions, and an Object Dynamics Control mechanism that combines 2D visual guidance with 3D trajectory embeddings to enable depth-aware, spatially coherent object manipulation. To support large-scale training and evaluation, we further construct RealCOD-25K, a comprehensive real-world dataset containing paired camera poses and object-level 3D trajectories across diverse indoor and outdoor scenes, addressing a key data gap in unified motion control. Extensive experiments and user studies show that SymphoMotion significantly outperforms existing methods in visual fidelity, camera controllability, and object-motion accuracy, establishing a new benchmark for unified motion control in video generation.Codes and data are publicly available at https://grenoble-zhang.github.io/SymphoMotion/.

Problem

• camera motion
• object dynamics
• coherent video generation
• motion control
• 3D trajectory

Innovation

• unified motion control
• camera trajectory
• object dynamics
• 3D trajectory embedding
• geometry-aware video generation