LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time

2026-08-12

AI Summary

This work addresses the challenge of real-time generation of high-quality, low-latency, long-duration human motion animations by proposing a streaming framework based on a 14-billion-parameter video diffusion Transformer. The approach employs a two-stage training strategy: first, a reference-anchored teacher-forcing scheme converts a bidirectional model into a block-wise causal autoregressive generator; second, block-level self-distillation compresses the sampling process to just three steps. To maintain appearance consistency with past poses under constant memory and latency constraints, the method introduces Pose-Retrieval Sink Attention. This is the first demonstration of stable real-time long-sequence generation with a billion-scale parameter model, achieving 19.63 FPS on dual H100 GPUs. The generated three-minute videos exhibit near-perfect identity consistency and high perceptual quality (IQA score of 4.047), significantly outperforming existing offline or degraded baselines.

Abstract

Pose-driven human animation synthesizes a video of a target person from a single reference image and a driving pose stream. Real-time generation is essential for interactive applications such as live streaming, telepresence, and virtual avatars, yet diffusion-based systems require minutes to hours per clip, precluding responsive interaction. We present LiveAnimate, to our knowledge the first animation system to combine real-time streaming with stable long-form generation at billion scale, built on a 14B-parameter video Diffusion Transformer (DiT). A two-stage training pipeline first adapts a pretrained bidirectional DiT into a block-causal autoregressive generator through Reference-Anchored Teacher-Forcing Adaptation, and then reduces the sampling budget to three steps through Block-wise Self-Forcing Distillation. To preserve appearance over extended streams, we introduce Pose-Retrieval Sink Attention (PR-Sink), a bounded KV-cache mechanism combining a Static Sink that permanently anchors the first generated block, a Dynamic Sink that holds a pose-retrieved historical block, and a three-slot Rolling Window. When a pose recurs, PR-Sink restores the relevant appearance context without retaining the entire sequence, so memory and per-block latency remain constant regardless of stream duration. Together with Ulysses sequence parallelism and operator fusion, these designs enable 19.63\,FPS streaming inference on two NVIDIA H100 GPUs. On a three-minute benchmark, LiveAnimate maintains nearly constant perceptual quality and identity from the first 30 seconds to the final minute (IQA 4.047 vs.\ 4.026), while prior systems degrade substantially or require hours of offline computation for the same rollout. These results establish a new operating point in quality, latency, and duration for interactive full-body animation.

Problem

• real-time animation
• long-form generation
• pose-driven human animation
• streaming inference
• appearance consistency

Innovation

• LiveAnimate
• Diffusion Transformer
• Pose-Retrieval Sink Attention
• real-time streaming animation
• block-causal autoregressive generation