UnReflectAnything: RGB-Only Highlight Removal by Rendering Synthetic Specular Supervision

2025-12-10

AI Summary

Specular highlights in natural and surgical images cause texture blurring and degrade geometric reasoning accuracy. Method: This paper introduces the first self-supervised specular highlight removal method for single RGB images. It innovatively designs a virtual highlight synthesis pipeline integrating monocular geometry estimation, Fresnel-aware shading, and stochastic lighting rendering to generate physically plausible highlight masks—enabling unpaired training on arbitrary real-world images. The network employs a frozen ViT encoder for multi-scale feature extraction, coupled with a lightweight highlight localization head and a token-level feature restoration module, jointly optimizing diffuse reflectance reconstruction and highlight map prediction in an end-to-end manner. Contribution/Results: Our approach achieves state-of-the-art performance across multiple cross-domain benchmarks (natural and surgical), significantly improving texture fidelity and geometric robustness on non-Lambertian surfaces under strong specular interference.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Security and Privacy: Data transparency and provenance

Abstract

Specular highlights distort appearance, obscure texture, and hinder geometric reasoning in both natural and surgical imagery. We present UnReflectAnything, an RGB-only framework that removes highlights from a single image by predicting a highlight map together with a reflection-free diffuse reconstruction. The model uses a frozen vision transformer encoder to extract multi-scale features, a lightweight head to localize specular regions, and a token-level inpainting module that restores corrupted feature patches before producing the final diffuse image. To overcome the lack of paired supervision, we introduce a Virtual Highlight Synthesis pipeline that renders physically plausible specularities using monocular geometry, Fresnel-aware shading, and randomized lighting which enables training on arbitrary RGB images with correct geometric structure. UnReflectAnything generalizes across natural and surgical domains where non-Lambertian surfaces and non-uniform lighting create severe highlights and it achieves competitive performance with state-of-the-art results on several benchmarks. Project Page: https://alberto-rota.github.io/UnReflectAnything/

Problem

• Removes specular highlights from single RGB images
• Uses synthetic supervision for training without paired data
• Generalizes across natural and surgical imagery domains

Innovation

• RGB-only framework removes highlights via highlight map prediction
• Virtual Highlight Synthesis renders specularities using monocular geometry
• Token-level inpainting module restores corrupted feature patches