FlashVideo:Flowing Fidelity to Detail for Efficient High-Resolution Video Generation

2025-02-07

AI Summary

High-resolution video generation faces a fundamental trade-off between fidelity and computational efficiency, particularly in single-stage DiT-based models, where large parameter counts and high numbers of function evaluations (NFEs) incur prohibitive computational overhead. To address this, we propose a two-stage cascaded diffusion framework: the first stage operates at low resolution using a large model with high NFE to ensure strong text alignment and motion fidelity; the second stage synthesizes high-resolution details via cross-resolution flow matching and feature alignment, achieving high fidelity with minimal NFE. We introduce the first stage-aware computational resource allocation mechanism, enabling real-time preview generation and significantly reducing end-to-end latency and compute cost. Our method achieves state-of-the-art performance across multiple benchmarks, reducing NFE by over 60%, accelerating inference by 2.3×, and maintaining superior text–video consistency and fine-grained visual realism.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

DiT diffusion models have achieved great success in text-to-video generation, leveraging their scalability in model capacity and data scale. High content and motion fidelity aligned with text prompts, however, often require large model parameters and a substantial number of function evaluations (NFEs). Realistic and visually appealing details are typically reflected in high resolution outputs, further amplifying computational demands especially for single stage DiT models. To address these challenges, we propose a novel two stage framework, FlashVideo, which strategically allocates model capacity and NFEs across stages to balance generation fidelity and quality. In the first stage, prompt fidelity is prioritized through a low resolution generation process utilizing large parameters and sufficient NFEs to enhance computational efficiency. The second stage establishes flow matching between low and high resolutions, effectively generating fine details with minimal NFEs. Quantitative and visual results demonstrate that FlashVideo achieves state-of-the-art high resolution video generation with superior computational efficiency. Additionally, the two-stage design enables users to preview the initial output before committing to full resolution generation, thereby significantly reducing computational costs and wait times as well as enhancing commercial viability .

Problem

• efficient high-resolution video generation
• reduce computational demands
• two-stage framework for fidelity

Innovation

• Two-stage framework
• Flow matching
• Efficient high-resolution generation