DeLux: Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data

2026-06-25

AI Summary

This work addresses the irreversible loss of structural information in RGB videos caused by intense illumination, proposing a novel cross-modal restoration paradigm that leverages neuromorphic event streams as structural priors for the first time. The method precisely detects and corrects localized photometric artifacts—such as glare, flicker, and overexposure—through a modular pipeline that fuses event and RGB data, trained on both synthetic artifacts and real-world scenes. Experimental results demonstrate significant superiority over existing RGB-only and event-guided HDR baselines on both synthetic and real automotive video datasets, achieving an MS-SSIM exceeding 0.99 and reducing artifact severity in real-world scenarios by up to 88%. This approach overcomes the fundamental limitation of conventional methods, which cannot recover structures completely occluded by saturation.

Abstract

Conventional RGB cameras suffer from lighting artifacts such as flare, glare, flicker, and overexposure, leading to irrecoverable information loss that necessitates computational restoration. However, existing approaches treat these problems in isolation, failing to recover structural details completely obscured by complex spatially discrete image degradations. In this paper, we propose a novel cross-modal restoration paradigm and present DeLux, a modular proof-of-concept pipeline that leverages neuromorphic event streams as a structural prior to guide the targeted detection and inpainting of lighting artifacts in RGB video. Validation on synthetic benchmarks and real-world automotive footage demonstrates that DeLux effectively suppresses local artifacts and restores affected regions. The proposed approach outperforms existing RGB-only baselines and event-guided HDR models, achieving an average MS-SSIM of over 0.99 across all artifact types and demonstrating up to an 88% reduction in artifact severity in real-world automotive footage. The synthetic artifact generation tools and curated real-world evaluation datasets are made publicly available to foster future research on cross-modal restoration.

Problem

• lighting artifacts
• image degradation
• structural detail recovery
• RGB video restoration
• cross-modal restoration

Innovation

• cross-modal restoration
• neuromorphic events
• lighting artifact inpainting
• structural prior
• video enhancement