Progressive Pose-Guided 4D Animal Reconstruction from Monocular Video

2026-06-30

AI Summary

This work addresses the challenge of 4D animal reconstruction from monocular videos, which is hindered by high species diversity, complex articulated motion, and the absence of a universal template, making it difficult for existing methods to balance generalization and fidelity. The authors propose a progressive test-time optimization framework based on 3D Gaussian splatting that decouples skeletal pose from non-rigid deformation to achieve high-fidelity reconstructions. Their approach incorporates a symmetry-aware temporal encoding to mitigate camera drift and introduces a part-conditioned deformation mechanism leveraging learnable part anchors and a skinning field, enabling accurate detail recovery even under coarse shape priors. Experiments demonstrate strong generalization across diverse animal species, with superior performance in geometric accuracy, temporal consistency, and visual quality compared to state-of-the-art methods, remaining robust even when the shape prior is severely mismatched.

Abstract

Reconstructing 4D animals from monocular videos is challenging due to large inter-species variation, complex articulations, and the lack of reliable templates. Existing approaches typically rely on either strict category-specific priors that restrict generalization, or unconstrained generative models that sacrifice input fidelity. To bridge this gap, we present a progressive test-time optimization framework built on 3D Gaussian Splatting for high-fidelity 4D animal reconstruction from a single video. Our key insight is that a coarse shape prior suffices when coupled with a progressive strategy that disentangles articulated pose from non-rigid deformation. Specifically, we employ a symmetry-aware temporal encoding that exploits bilateral cues while absorbing camera estimation drift and a part-conditioned deformation mechanism guided by learnable part anchors and a learnable skinning field. Extensive experiments demonstrate that our approach generalizes robustly across diverse species, achieving superior geometric accuracy, temporal consistency, and visual fidelity compared to existing baselines, even under severe prior mismatch.

Problem

• 4D animal reconstruction
• monocular video
• articulated pose
• non-rigid deformation
• shape prior

Innovation

• progressive optimization
• 3D Gaussian Splatting
• pose-guided deformation
• monocular 4D reconstruction
• cross-species generalization