Problem
Research questions and friction points this paper is trying to address.
streaming audio-video generation
real-time interactive worlds
evaluation benchmark
StreamAV-Bench: A Comprehensive Benchmark for Streaming Audio-Video Generation
🤖 AI Summary
本文针对实时互动世界的流式音视频生成问题，提出了StreamAV-Bench基准，通过统一评估框架和32个细粒度维度的测试案例来评估13个代表性系统。
📝 Abstract
Recent advancements in generative models are pushing video generation toward unbounded streaming audio-video generation for real-time interactive worlds. However, existing benchmarks primarily evaluate completed sequences and struggle to capture streaming properties. To bridge this gap, we introduce StreamAV-Bench, the first comprehensive benchmark tailored for streaming audio-video generation. StreamAV-Bench establishes a unified evaluation framework, including the progressive track for instruction adherence and long-horizon stability, and the interactive track for interactive response and state retention and reuse. With expert-verified evaluation cases across 32 fine-grained dimensions, we conduct an extensive evaluation of 13 representative systems. Our analysis reveals that current models suffer from temporal drift in progressive generation and responsiveness bottlenecks during interactive control. Based on a comprehensive failure analysis, we share insights to advance the development of native joint audio-video streaming models.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Streaming Audio-Video Generation
Unified Evaluation Framework
Temporal Drift
Interactive Response
State Retention and Reuse
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
