DSAR: Dual-Stream Autoregressive Modeling of Temporal Cloth Dynamics for Photorealistic Animatable Avatars

2026-08-11

AI Summary

Existing methods for generating animatable avatars from RGB videos often produce overly smoothed appearances or artifacts during pose extrapolation due to neglecting the temporal causality of cloth dynamics. This work proposes the first dual-stream autoregressive framework that explicitly models temporal causality: a geometry stream propagates surface displacements from the previous frame, while a state stream integrates current features with historical hidden states via a memory bank, enhanced by motion-adaptive aggregation to capture spatially varying dynamics. By decoupling geometry propagation from state evolution, the approach overcomes the limitations of conventional instantaneous pose modeling. It achieves significantly improved rendering quality, temporal consistency, and generalization to out-of-distribution motions across multiple challenging datasets, yielding more realistic cloth dynamics.

Abstract

Creating photorealistic and temporally coherent animatable human avatars from RGB videos remains challenging. Current methods struggle to capture realistic cloth dynamics, producing over-smoothed appearance or severe artifacts on out-of-distribution poses. This limitation stems from a fundamental oversight: existing approaches neglect the temporal causality inherent in cloth physics, where current states emerge from previous states through temporal evolution rather than instantaneous skeletal configurations alone. Without explicit modeling of this causal structure, networks learn pose-appearance correlations instead of motion evolution, leading to poor generalization. We introduce a dual-stream autoregressive framework that explicitly models both observable geometric information and implicit internal state. The geometric stream propagates surface displacement from the previous frame, while the state stream fuses current features with historical states retrieved from a memory bank. Motion-adaptive aggregation handles spatially-varying dynamics, and adaptive regularization balances smoothness with flexibility. Experiments on challenging datasets demonstrate significant improvements in rendering quality, temporal consistency, and generalization to motion patterns beyond training distributions, validating that dual-stream temporal modeling enables realistic cloth dynamics.

Problem

• cloth dynamics
• temporal causality
• animatable avatars
• photorealistic rendering
• out-of-distribution generalization

Innovation

• dual-stream autoregressive modeling
• temporal cloth dynamics
• animatable avatars
• motion-adaptive aggregation
• memory-augmented state propagation