InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars

2026-06-22

AI Summary

Existing real-time streaming methods for virtual human video generation struggle to simultaneously maintain long-term visual temporal consistency and accurately perceive user intent. This work proposes a real-time framework capable of generating videos of unlimited duration, leveraging autoregressive distillation to enhance inference efficiency. It introduces an innovative fusion of short- and long-term visual memory mechanisms with a reasoning-and-response module, complemented by a state-recurrent strategy and a cache-switching mechanism. This design enables high visual consistency while effectively aligning with complex user intentions. Experimental results demonstrate that the proposed method significantly outperforms current state-of-the-art approaches across diverse scenarios, achieving—for the first time in real-time streaming generation—concurrent long-term visual coherence and responsive interactive intent alignment.

Abstract

Recent diffusion-based models have enabled realistic audio-driven avatar generation in real-time streaming. However, existing approaches struggle to maintain visual temporal consistency and fail to explicitly perceive user intent in complex interactive streaming scenarios. To address these challenges, we propose InteractiveAvatar, a real-time infinite-streaming video generation framework that supports visually consistent avatar video generation and intent-aware interactions. With autoregressive distillation, InteractiveAvatar achieves real-time str-eaming generation of human avatars over arbitrarily long durations. For visual consistency, we introduce a Long-Short Visual Memory (LSVM) mechanism that flexibly compresses historical visual information into compact tokens, preserving both short-range coherence and long-term consistency. To generate avatars with speeches and actions aligned with user intent, we propose a Reasoning-Reaction Module (RRM), which incorporates a State-Cycling strategy and a Cache-Switching mechanism. Extensive experimental results over diverse scenarios demonstrate that our method achieves state-of-the-art visual consistency in long-duration generation, while enabling complex user-avatar interaction in real time.

Problem

• temporal consistency
• user intent
• real-time streaming
• avatar generation
• interactive avatars

Innovation

• real-time streaming
• visual temporal consistency
• intent-aware interaction
• Long-Short Visual Memory
• Reasoning-Reaction Module