GA-Drive: Geometry-Appearance Decoupled Modeling for Free-viewpoint Driving Scene Generatio

2026-02-24

AI Summary

Existing methods struggle to generate driving scenes that are simultaneously free-viewpoint, high-fidelity, and editable, thereby limiting the training and evaluation of end-to-end autonomous driving systems. This work proposes a geometry-appearance disentangled generative framework that synthesizes novel-view pseudo-images guided by explicit geometric representations. By integrating video diffusion models with video-to-video editing techniques, the approach enables controllable appearance synthesis while preserving geometric consistency. It further supports cross-trajectory appearance editing, enhancing generation flexibility without compromising structural coherence. Experimental results demonstrate significant improvements over state-of-the-art methods on key metrics including NTA-IoU, NTL-IoU, and FID, yielding more realistic and structurally accurate driving scenes.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

A free-viewpoint, editable, and high-fidelity driving simulator is crucial for training and evaluating end-to-end autonomous driving systems. In this paper, we present GA-Drive, a novel simulation framework capable of generating camera views along user-specified novel trajectories through Geometry-Appearance Decoupling and Diffusion-Based Generation. Given a set of images captured along a recorded trajectory and the corresponding scene geometry, GA-Drive synthesizes novel pseudo-views using geometry information. These pseudo-views are then transformed into photorealistic views using a trained video diffusion model. In this way, we decouple the geometry and appearance of scenes. An advantage of such decoupling is its support for appearance editing via state-of-the-art video-to-video editing techniques, while preserving the underlying geometry, enabling consistent edits across both original and novel trajectories. Extensive experiments demonstrate that GA-Drive substantially outperforms existing methods in terms of NTA-IoU, NTL-IoU, and FID scores.

Problem

• free-viewpoint generation
• driving scene simulation
• geometry-appearance decoupling
• autonomous driving
• photorealistic rendering

Innovation

• Geometry-Appearance Decoupling
• Diffusion-Based Generation
• Free-viewpoint Synthesis
• Driving Scene Simulation
• Video-to-Video Editing