AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing

2026-03-27

AI Summary

Existing generative video models for synthesizing autonomous driving scenes under extreme weather conditions rely heavily on large-scale data, while 3D-based editing approaches are limited by the entanglement of geometry and lighting and the high cost of per-scene optimization. This work proposes a feedforward, 3D-aware weather editing framework that explicitly decouples geometry and lighting through a G-Buffer dual-channel mechanism: the geometry channel models physical surface interactions, while the lighting channel captures light transport and enables dynamic, 3D-localized relighting. To our knowledge, this is the first method to achieve explicit geometry–lighting disentanglement without per-scene optimization, allowing fine-grained control over physical parameters. The approach significantly reduces data dependency while preserving visual realism and structural consistency, offering an efficient and physically plausible weather augmentation engine for autonomous driving applications.

Abstract

Generative video models have significantly advanced the photorealistic synthesis of adverse weather for autonomous driving; however, they consistently demand massive datasets to learn rare weather scenarios. While 3D-aware editing methods alleviate these data constraints by augmenting existing video footage, they are fundamentally bottlenecked by costly per-scene optimization and suffer from inherent geometric and illumination entanglement. In this work, we introduce AutoWeather4D, a feed-forward 3D-aware weather editing framework designed to explicitly decouple geometry and illumination. At the core of our approach is a G-buffer Dual-pass Editing mechanism. The Geometry Pass leverages explicit structural foundations to enable surface-anchored physical interactions, while the Light Pass analytically resolves light transport, accumulating the contributions of local illuminants into the global illumination to enable dynamic 3D local relighting. Extensive experiments demonstrate that AutoWeather4D achieves comparable photorealism and structural consistency to generative baselines while enabling fine-grained parametric physical control, serving as a practical data engine for autonomous driving.

Problem

• autonomous driving
• weather synthesis
• 3D-aware editing
• geometry-illumination entanglement
• data efficiency

Innovation

• G-buffer Dual-pass Editing
• geometry-illumination decoupling
• 3D-aware video editing
• parametric physical control
• autonomous driving simulation