LightsOut: Diffusion-based Outpainting for Enhanced Lens Flare Removal

2025-10-17

AI Summary

Lens flare severely degrades image quality, hindering downstream vision tasks such as object detection and autonomous driving. Existing single-image flare removal (SIFR) methods suffer significant performance degradation when the flare source lies outside the frame or is occluded. To address this, we propose the first diffusion-based out-of-frame flare removal framework. Our method jointly optimizes the physical parameters—namely, source location, intensity, and geometric properties—via a physics-guided multi-task regression, and leverages a LoRA-finetuned diffusion model to synthesize structurally coherent and photometrically plausible flare completions for off-frame sources. Crucially, it integrates diffusion priors with explicit physical illumination modeling—enabling plug-and-play deployment as a preprocessing module without retraining. Experiments demonstrate consistent performance gains across mainstream SIFR methods under challenging scenarios (e.g., incomplete or missing flare sources), achieving average improvements of +2.1 dB in PSNR and +0.032 in SSIM.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Lens flare significantly degrades image quality, impacting critical computer vision tasks like object detection and autonomous driving. Recent Single Image Flare Removal (SIFR) methods perform poorly when off-frame light sources are incomplete or absent. We propose LightsOut, a diffusion-based outpainting framework tailored to enhance SIFR by reconstructing off-frame light sources. Our method leverages a multitask regression module and LoRA fine-tuned diffusion model to ensure realistic and physically consistent outpainting results. Comprehensive experiments demonstrate LightsOut consistently boosts the performance of existing SIFR methods across challenging scenarios without additional retraining, serving as a universally applicable plug-and-play preprocessing solution. Project page: https://ray-1026.github.io/lightsout/

Problem

• Removing lens flare artifacts from degraded images
• Handling incomplete off-frame light sources in flare removal
• Enhancing existing flare removal methods without retraining

Innovation

• Diffusion-based outpainting reconstructs off-frame light sources
• Multitask regression module ensures physically consistent results
• LoRA fine-tuned model serves as plug-and-play preprocessing solution