Shift-Window Meets Dual Attention: A Multi-Model Architecture for Specular Highlight Removal

2025-12-04

AI Summary

Specular highlights are ubiquitous in real-world scenes and severely degrade performance across vision tasks. Existing approaches either rely on CNNs to model local details or employ Transformers to capture global dependencies, yet struggle to simultaneously achieve fine-grained suppression of multi-scale highlights and effective long-range modeling. This paper proposes a multi-scale specular highlight removal framework that unifies local and global representation learning. We introduce an omnidirectional pixel offset module and an adaptive region-aware hybrid-domain dual-attention convolution (HDDAConv), which jointly integrate convolutional and dual-attention mechanisms within a coarse-to-fine architecture. Additionally, we propose the OAIBlock and a sliding-window strategy to balance accuracy and computational efficiency. Our method achieves significant improvements over state-of-the-art methods across three benchmark tasks and six material categories. The source code is publicly available.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Inevitable specular highlights in practical environments severely impair the visual performance, thus degrading the task effectiveness and efficiency. Although there exist considerable methods that focus on local information from convolutional neural network models or global information from transformer models, the single-type model falls into a modeling dilemma between local fine-grained details and global long-range dependencies, thus deteriorating for specular highlights with different scales. Therefore, to accommodate specular highlights of all scales, we propose a multi-model architecture for specular highlight removal (MM-SHR) that effectively captures fine-grained features in highlight regions and models long-range dependencies between highlight and highlight-free areas. Specifically, we employ convolution operations to extract local details in the shallow layers of MM-SHR, and utilize the attention mechanism to capture global features in the deep layers, ensuring both operation efficiency and removal accuracy. To model long-range dependencies without compromising computational complexity, we utilize a coarse-to-fine manner and propose Omni-Directional Attention Integration Block(OAIBlock) and Adaptive Region-Aware Hybrid-Domain Dual Attention Convolutional Network(HDDAConv) , which leverage omni-directiona pixel-shifting and window-dividing operations at the raw features to achieve specular highlight removal. Extensive experimental results on three benchmark tasks and six types of surface materials demonstrate that MM-SHR outperforms state-of-the-art methods in both accuracy and efficiency for specular highlight removal. The implementation will be made publicly available at https://github.com/Htcicv/MM-SHR.

Problem

• Removes specular highlights across different scales effectively
• Captures fine-grained details and global dependencies simultaneously
• Improves accuracy and efficiency in highlight removal tasks

Innovation

• Multi-model architecture combining convolution and attention
• Omni-directional attention integration for long-range dependencies
• Adaptive hybrid-domain dual attention for highlight removal