Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environments

2026-04-12

AI Summary

This work addresses the performance degradation of object detection in hazy environments caused by image degradation and semantic ambiguity. Departing from conventional image enhancement approaches, it pioneers the integration of language prompts with CLIP to semantically guide the reinforcement of weakened visual features. The study introduces an Approximate Mutual Exclusivity (AME) mechanism to assign reliable weights to the classification loss and further proposes a Fine-tuned AME (FAME) strategy to adaptively optimize weight balancing during training. Evaluated on HazyCOCO—a large-scale synthetic dataset curated by the authors—the proposed method achieves significant improvements in detection accuracy, establishing state-of-the-art performance under haze conditions.

Technology Category

• Computer Vision: Language and Vision
• Search and Optimization: Learning to Search
• Machine Learning: Hardware-aware ML

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Object detection in hazy environments is challenging because degraded objects are nearly invisible and their semantics are weakened by environmental noise, making it difficult for detectors to identify. Common approaches involve image enhancement to boost weakened semantics, but these methods are limited by the instability of enhanced modules. This paper proposes a novel solution by employing language prompts to enhance weakened semantics without image enhancement. Specifically, we design Approximation of Mutual Exclusion (AME) to provide credible weights for Cross-Entropy Loss, resulting in CLIP-guided Cross-Entropy Loss (CLIP-CE). The provided weights assess the semantic weakening of objects. Through the backpropagation of CLIP-CE, weakened semantics are enhanced, making degraded objects easier to detect. In addition, we present Fine-tuned AME (FAME) which adaptively fine-tunes the weight of AME based on the predicted confidence. The proposed FAME compensates for the imbalanced optimization in AME. Furthermore, we present HazyCOCO, a large-scale synthetic hazy dataset comprising 61258 images. Experimental results demonstrate that our method achieves state-of-the-art performance. The code and dataset will be released.

Problem

• object detection
• hazy environments
• semantic weakening
• image degradation
• CLIP

Innovation

• language prompt
• CLIP-guided loss
• semantic enhancement
• hazy object detection
• mutual exclusion approximation