Robust Onion: Peeling Open Vocab Object Detectors Under Noise

2026-06-25

AI Summary

The robustness mechanisms of open-vocabulary object detectors under real-world noise remain poorly understood. This work systematically investigates the causes of feature collapse and robustness degradation through controlled synthetic visual degradations. It reveals, for the first time, that robustness is primarily governed by the image domain rather than annotations, and demonstrates that shared visual backbones lead to similar robustness characteristics across models. Building on these insights, the authors propose lightweight NN and TK0 plug-in modules that significantly enhance robustness on real-world datasets—including BDD100K, WiderFace, and VisDrone—while introducing only 1/96 of the end-to-end trainable parameters. The framework also provides a unified explanation for empirical observations reported in prior studies.

Abstract

The impact of real-world noise on Open Vocabulary Object Detectors (OV-ODs) remains poorly understood due to their architectural complexity. We present our comprehensive analysis Robust Onion, an empirical study that uses controlled synthetic visual degradations to peel OV-ODs layer-by-layer, revealing how, why, and where robustness degrades, systematically analyzing feature collapse. Our findings reveal that models with similar vision backbones exhibit comparable robustness, driven by similar feature collapse at similar layers, while factors such as pretraining strategy, architectural nuances, and caption supervision contribute little. Robustness is primarily governed by the image domain rather than annotations, explaining the similar robustness impact on COCO and LVIS, and why datasets like ODinW-13 can give an impression of inflated robustness due to large, isolated objects. Finally, we validate our insights by improving robustness on real-world BDD100K, WiderFace, and VisDRONE via our lightweight plug-and-play NN & TK0 approach, using 96x fewer trainable parameters than end-to-end training. We also explain the prior works' robustness observations.

Problem

• Open Vocabulary Object Detection
• Robustness
• Noise
• Feature Collapse
• Visual Degradation

Innovation

• Open Vocabulary Object Detection
• Robustness Analysis
• Feature Collapse
• Synthetic Visual Degradation
• Plug-and-Play Enhancement