ClimaOoD: Improving Anomaly Segmentation via Physically Realistic Synthetic Data

2025-12-02

AI Summary

Anomaly segmentation suffers from both the scarcity of real-world anomalous samples and the physical implausibility and contextual inconsistency of synthetic anomalies, severely limiting model generalization in open-world driving scenarios. To address this, we propose ClimaDrive—a novel framework that unifies structural guidance for multi-weather image generation with prompt-driven anomaly region inpainting, producing semantically coherent, weather-diverse, and physically plausible anomalous driving scenes. Based on this, we construct ClimaOoD, a large-scale, multi-weather out-of-distribution anomaly detection benchmark. Our method integrates semantic-map-guided diffusion models, multi-condition weather modeling, and fine-grained prompt engineering. Extensive evaluation across four state-of-the-art anomaly segmentation models demonstrates substantial improvements: ClimaOoD-trained models achieve consistent gains in AUROC, AP, and FPR95. Notably, RbA’s FPR95 on Fishyscapes LAF drops from 3.97 to 3.52, confirming significantly enhanced robustness against diverse weather-induced anomalies.

Technology Category

• Computer Vision: Segmentation
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Anomaly segmentation seeks to detect and localize unknown or out-of-distribution (OoD) objects that fall outside predefined semantic classes a capability essential for safe autonomous driving. However, the scarcity and limited diversity of anomaly data severely constrain model generalization in open-world environments. Existing approaches mitigate this issue through synthetic data generation, either by copy-pasting external objects into driving scenes or by leveraging text-to-image diffusion models to inpaint anomalous regions. While these methods improve anomaly diversity, they often lack contextual coherence and physical realism, resulting in domain gaps between synthetic and real data. In this paper, we present ClimaDrive, a semantics-guided image-to-image framework for synthesizing semantically coherent, weather-diverse, and physically plausible OoD driving data. ClimaDrive unifies structure-guided multi-weather generation with prompt-driven anomaly inpainting, enabling the creation of visually realistic training data. Based on this framework, we construct ClimaOoD, a large-scale benchmark spanning six representative driving scenarios under both clear and adverse weather conditions. Extensive experiments on four state-of-the-art methods show that training with ClimaOoD leads to robust improvements in anomaly segmentation. Across all methods, AUROC, AP, and FPR95 show notable gains, with FPR95 dropping from 3.97 to 3.52 for RbA on Fishyscapes LAF. These results demonstrate that ClimaOoD enhances model robustness, offering valuable training data for better generalization in open-world anomaly detection.

Problem

• Generates realistic synthetic data for anomaly segmentation
• Addresses lack of diversity in anomaly data for autonomous driving
• Improves model generalization in open-world environments

Innovation

• Semantics-guided image-to-image framework for realistic synthetic data
• Unifies multi-weather generation with prompt-driven anomaly inpainting
• Creates physically plausible OoD driving data for robust training