Self-Aware Object Detection via Degradation Manifolds

2026-02-20

AI Summary

This work addresses the silent failure of modern object detectors under image degradations—such as blur, noise, and adverse weather—stemming from their inability to self-assess whether inputs lie within their effective operating regime. The authors propose a self-aware mechanism that requires neither degradation labels nor explicit density modeling. By constructing a degradation manifold, the method organizes feature representations according to degradation type rather than semantic content. A lightweight embedding head, trained with multi-level contrastive learning, encourages intra-degradation clustering and inter-degradation separation. Clean prototypes define nominal operating points, enabling degradation awareness decoupled from detection confidence. The approach demonstrates strong clean-versus-degraded separability, architectural compatibility, and robust generalization across synthetic corruptions, zero-shot cross-dataset transfer, and natural distribution shifts induced by real-world weather conditions.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Hardware-aware ML
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Object detectors achieve strong performance under nominal imaging conditions but can fail silently when exposed to blur, noise, compression, adverse weather, or resolution changes. In safety-critical settings, it is therefore insufficient to produce predictions without assessing whether the input remains within the detector's nominal operating regime. We refer to this capability as self-aware object detection. We introduce a degradation-aware self-awareness framework based on degradation manifolds, which explicitly structure a detector's feature space according to image degradation rather than semantic content. Our method augments a standard detection backbone with a lightweight embedding head trained via multi-layer contrastive learning. Images sharing the same degradation composition are pulled together, while differing degradation configurations are pushed apart, yielding a geometrically organized representation that captures degradation type and severity without requiring degradation labels or explicit density modeling. To anchor the learned geometry, we estimate a pristine prototype from clean training embeddings, defining a nominal operating point in representation space. Self-awareness emerges as geometric deviation from this reference, providing an intrinsic, image-level signal of degradation-induced shift that is independent of detection confidence. Extensive experiments on synthetic corruption benchmarks, cross-dataset zero-shot transfer, and natural weather-induced distribution shifts demonstrate strong pristine-degraded separability, consistent behavior across multiple detector architectures, and robust generalization under semantic shift. These results suggest that degradation-aware representation geometry provides a practical and detector-agnostic foundation.

Problem

• self-aware object detection
• degradation manifolds
• distribution shift
• nominal operating regime
• image degradation

Innovation

• self-aware object detection
• degradation manifolds
• contrastive learning
• representation geometry
• distribution shift