Driving-Video Dehazing with Non-Aligned Regularization for Safety Assistance

2024-05-16

Computer Vision and Pattern Recognition

AI Summary

This work addresses the critical challenge in real-world driving video dehazing—namely, the scarcity of strictly aligned hazy/clear video pairs. To this end, we propose an end-to-end video dehazing framework tailored for misaligned data. Our method introduces three key innovations: (1) an adaptive sliding-window mechanism for selecting non-aligned reference frames; (2) a flow-guided cosine attention sampling module to enhance motion-consistent feature modeling; and (3) a deformable cosine attention fusion module to improve spatiotemporal feature aggregation. Evaluated on our newly constructed GoProHazy driving video dataset—featuring realistic haze, dynamic motion, and diverse weather conditions—our approach significantly outperforms existing state-of-the-art methods. Notably, it achieves superior dehazing quality and enhanced driving-assistance reliability, especially in highly dynamic scenes and challenging meteorological conditions.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Mixture of Experts (MoE)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Security and Privacy: Data transparency and provenance

Abstract

Real driving-video dehazing poses a significant challenge due to the inherent difficulty in acquiring precisely aligned hazy/clear video pairs for effective model training, especially in dynamic driving scenarios with unpredictable weather conditions. In this paper, we propose a pioneering approach that addresses this challenge through a nonaligned regularization strategy. Our core concept involves identifying clear frames that closely match hazy frames, serving as references to supervise a video dehazing network. Our approach comprises two key components: reference matching and video dehazing. Firstly, we introduce a non-aligned reference frame matching module, leveraging an adaptive sliding window to match high-quality reference frames from clear videos. Video dehazing incorporates flow-guided cosine attention sampler and deformable cosine attention fusion modules to enhance spatial multi-frame alignment and fuse their improved information. To validate our approach, we collect a GoProHazy dataset captured effortlessly with GoPro cameras in diverse rural and urban road environments. Extensive experiments demonstrate the superiority of the proposed method over current state-of-the-art methods in the challenging task of real driving-video dehazing. Project page.

Problem

• Real driving-video dehazing without aligned hazy/clear pairs.
• Dynamic weather conditions complicate video dehazing in driving scenarios.
• Proposing non-aligned regularization for effective video dehazing.

Innovation

• Non-aligned regularization strategy for video dehazing
• Adaptive sliding window for reference frame matching
• Flow-guided cosine attention for spatial alignment