Follow Your Track: Precise Skeleton Animation Controlled by 3D Trajectories

2026-06-23

AI Summary

Existing 4D generation methods suffer from limitations in animation quality, temporal consistency, and controllability, primarily due to reliance on computationally expensive dense representations or insufficient fine-grained spatiotemporal control. This work proposes ACT, a framework that leverages lightweight skeletons as structured representations and employs 3D point trajectories extracted from monocular videos as explicit motion guidance to enable topology-agnostic, precise skeletal animation control. The core innovation lies in a routing trajectory injector that integrates prior-guided hard routing, a global routing mechanism, and local window cross-attention to achieve robust mapping from trajectories to joints, thereby enhancing whole-body motion awareness and micro-temporal alignment. Experiments demonstrate that ACT significantly outperforms existing approaches in animation fidelity, temporal consistency, and controllability, enabling high-quality, long-duration 4D animation synthesis.

Abstract

4D generation aims to animate 3D objects with realistic motion, holding great promise for applications. Existing methods typically decouple 3D asset generation from motion synthesis: acquire a 3D asset, prepare a structural representation like mesh and Gaussians, and synthesize motion from text or video control signals. However, dense mesh and Gaussian representations incur high computational costs and are prone to temporal artifacts, limiting animation quality and duration to only short clips. Meanwhile, text lacks fine-grained spatial and temporal details such as timing and coordination, while video entangles motion with appearance and background. Together, these limitations result in 4D animations that suffer from poor temporal consistency, wrong identification, and limited controllability. We address these issues with \texttt{ACT}, a trajectory-conditioned framework for topology-general skeletal animation. ACT uses skeletons as a compact structured and compute-efficient representation and 3D point trajectories from monocular video as explicit motion guidance which provide detailed motion patterns without appearance entanglement. At the core of ACT is a Routed Trajectory Injector, which achieves accurate and robust trajectory-to-joint transfer through three complementary designs: prior-guided hard routing establishes precise skeleton-to-mesh correspondences, global routing enables holistic joint-track interaction for full-body motion awareness, and local windowed cross-attention enforces fine-grained temporal alignment, improving micro-timing and reducing motion misalignment across varying motion rates. Extensive experiments demonstrate that \texttt{ACT} significantly outperforms existing methods in fidelity and temporal consistency.

Problem

• 4D generation
• skeletal animation
• temporal consistency
• motion controllability
• 3D trajectories

Innovation

• skeletal animation
• 3D trajectories
• trajectory-conditioned generation
• temporal consistency
• 4D animation