Semantic-Aware, Physics-Informed, Geometry-Grounded Weather Video Synthesis

2026-06-27

AI Summary

Existing methods for weather video synthesis struggle to simultaneously achieve appearance diversity and physically plausible dynamic control. To address this challenge, this work proposes a triply conditioned guidance framework that decomposes weather synthesis into three independent yet stable control signals: semantics, dynamics, and geometry. Semantic-aware conditioning anchors visual style, a Gaussian particle field simulates physical processes such as gravity, wind, and turbulence, and geometric alignment ensures spatial consistency. This approach enables the generation of weather videos that are visually diverse, dynamically detailed, and physically coherent, significantly enhancing the robustness of autonomous driving semantic segmentation models under adverse weather conditions.

Abstract

Weather synthesis aims to add weather effects to input videos while preserving scene identity, structure, and motion. The key limitation of existing methods is the lack of diversity in weather appearance and effective control over weather dynamics (e.g., temporal evolution and particle motion). Most approaches rely on text prompts, which are inherently underspecified and often fail to produce detailed weather characteristics. Additionally, general-purpose video editors optimized for clean and aesthetic outputs tend to suppress heavy weather phenomena, making dense particle effects difficult to generate. To address these, we propose a Semantic-Aware, Physics-Informed, and Geometry-Grounded framework that steers an off-the-shelf video editor to synthesize diverse global appearances and detailed particle dynamics. We factorize the synthesis into three conditional signals, so that each provides a distinct and stable source of guidance: semantics specifies what the weather should look like, dynamics governs how it evolves over time, and geometry determines where it should appear in the scene. Specifically, we introduce (1) semantic-aware appearance anchoring to establish the target appearance from scene semantics and user input; (2) physics-informed dynamic simulation to generate particle effects by simulating a Gaussian-represented particle field under gravity, wind, and turbulence; and (3) geometry-grounded video synthesis to align the simulated particles with target scene geometry and synthesize the final video. Experiments demonstrate that our method produces diverse, physically and visually realistic weather effects. Furthermore, we show that our synthesized data significantly improves the robustness of autonomous driving semantic segmentation under adverse weather conditions. Project page: https://jumponthemoon.github.io/w-crafter/.

Problem

• weather video synthesis
• weather dynamics
• particle effects
• appearance diversity
• adverse weather simulation

Innovation

• Semantic-Aware
• Physics-Informed
• Geometry-Grounded
• Weather Video Synthesis
• Particle Dynamics