AutoScape: Geometry-Consistent Long-Horizon Scene Generation

2025-10-23

AI Summary

To address geometric inconsistency and poor temporal coherence in long-horizon autonomous driving video generation, this paper proposes a two-stage RGB-D diffusion framework. First, RGB and depth modalities are jointly modeled in a shared latent space, with explicit point-cloud representation enforcing scene geometry; a deformation-consistency guidance mechanism ensures sparse keyframes that are strictly geometrically consistent. Second, these keyframes serve as anchors to drive a video diffusion model for dense frame interpolation. This is the first method enabling end-to-end, geometrically consistent synthesis of long-duration (>20 seconds) driving videos. It achieves new state-of-the-art performance, outperforming prior methods by 48.6% on long-horizon FID and 43.0% on FVD, significantly enhancing visual realism and structural stability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper proposes AutoScape, a long-horizon driving scene generation framework. At its core is a novel RGB-D diffusion model that iteratively generates sparse, geometrically consistent keyframes, serving as reliable anchors for the scene's appearance and geometry. To maintain long-range geometric consistency, the model 1) jointly handles image and depth in a shared latent space, 2) explicitly conditions on the existing scene geometry (i.e., rendered point clouds) from previously generated keyframes, and 3) steers the sampling process with a warp-consistent guidance. Given high-quality RGB-D keyframes, a video diffusion model then interpolates between them to produce dense and coherent video frames. AutoScape generates realistic and geometrically consistent driving videos of over 20 seconds, improving the long-horizon FID and FVD scores over the prior state-of-the-art by 48.6% and 43.0%, respectively.

Problem

• Generating long-horizon geometrically consistent driving scenes
• Maintaining geometric consistency across sparse keyframes and dense videos
• Improving realism and coherence in extended driving video generation

Innovation

• RGB-D diffusion model generates sparse keyframes
• Shared latent space handles image and depth
• Video diffusion model interpolates dense video frames