From Pixel to Mask: A Survey of Out-of-Distribution Segmentation

2025-08-13

AI Summary

This work addresses pixel-level out-of-distribution (OoD) object localization and segmentation in autonomous driving, aiming to enhance perception robustness for safety-critical systems. We propose the first four-category taxonomy for OoD pixel segmentation: test-time adaptation, supervised anomaly training, reconstruction-driven methods, and large-model-enabled paradigms—systematically analyzing their technical principles, performance limits, and scenario applicability. Our methodological contributions include the novel integration of anomaly exposure training, test-time adaptive reconstruction, and vision-language prior guidance, embedded within a unified evaluation framework. Extensive experiments benchmark 12 state-of-the-art approaches across four standard datasets, revealing key bottlenecks and identifying three critical future directions: scalable architecture design, multimodal anomaly modeling, and lightweight deployment.

Technology Category

• Computer Vision: Segmentation
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Out-of-distribution (OoD) detection and segmentation have attracted growing attention as concerns about AI security rise. Conventional OoD detection methods identify the existence of OoD objects but lack spatial localization, limiting their usefulness in downstream tasks. OoD segmentation addresses this limitation by localizing anomalous objects at pixel-level granularity. This capability is crucial for safety-critical applications such as autonomous driving, where perception modules must not only detect but also precisely segment OoD objects, enabling targeted control actions and enhancing overall system robustness. In this survey, we group current OoD segmentation approaches into four categories: (i) test-time OoD segmentation, (ii) outlier exposure for supervised training, (iii) reconstruction-based methods, (iv) and approaches that leverage powerful models. We systematically review recent advances in OoD segmentation for autonomous-driving scenarios, identify emerging challenges, and discuss promising future research directions.

Problem

• Surveying OoD segmentation methods for AI security
• Addressing pixel-level localization in OoD detection
• Enhancing autonomous driving via robust OoD segmentation

Innovation

• Pixel-level OoD segmentation for spatial localization
• Four-category classification of OoD segmentation methods
• Focus on autonomous-driving safety applications