WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion

2025-04-18

AI Summary

Existing LiDAR point cloud simulators suffer from limited weather diversity (supporting only single-weather conditions) and low physical fidelity, while real-world adverse-weather point clouds are scarce. Method: This paper proposes the first unified, diverse weather-aware diffusion-based generation framework for physically consistent point cloud synthesis under rain, fog, snow, and other conditions. It (1) introduces Spider Mamba to model spatiotemporal dependencies inherent in LiDAR’s rotational scanning pattern; (2) incorporates a language-guided contrastive controller for fine-grained semantic control over weather attributes; and (3) designs a latent feature aligner to transfer priors from real-world adverse-weather scenes. Contribution/Results: The generated point clouds achieve significantly improved physical fidelity and diversity, enabling the construction of the mini-weather benchmark dataset. Downstream 3D detection and segmentation models trained with synthetic data show mAP gains of +5.2% to +8.7% under adverse weather conditions.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Web data generation and simulation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

3D scene perception demands a large amount of adverse-weather LiDAR data, yet the cost of LiDAR data collection presents a significant scaling-up challenge. To this end, a series of LiDAR simulators have been proposed. Yet, they can only simulate a single adverse weather with a single physical model, and the fidelity of the generated data is quite limited. This paper presents WeatherGen, the first unified diverse-weather LiDAR data diffusion generation framework, significantly improving fidelity. Specifically, we first design a map-based data producer, which can provide a vast amount of high-quality diverse-weather data for training purposes. Then, we utilize the diffusion-denoising paradigm to construct a diffusion model. Among them, we propose a spider mamba generator to restore the disturbed diverse weather data gradually. The spider mamba models the feature interactions by scanning the LiDAR beam circle or central ray, excellently maintaining the physical structure of the LiDAR data. Subsequently, following the generator to transfer real-world knowledge, we design a latent feature aligner. Afterward, we devise a contrastive learning-based controller, which equips weather control signals with compact semantic knowledge through language supervision, guiding the diffusion model to generate more discriminative data. Extensive evaluations demonstrate the high generation quality of WeatherGen. Through WeatherGen, we construct the mini-weather dataset, promoting the performance of the downstream task under adverse weather conditions. Code is available: https://github.com/wuyang98/weathergen

Problem

• Generates diverse adverse-weather LiDAR data with high fidelity
• Overcomes single-weather simulation limitations in existing LiDAR simulators
• Enhances 3D scene perception under adverse weather conditions

Innovation

• Map-based data producer for diverse-weather training
• Spider mamba generator for LiDAR structure preservation
• Contrastive learning controller with language supervision