Ambient-robust Inverse Rendering using Active RGB-NIR Imaging

2026-05-28

AI Summary

This work addresses the challenge of accurately recovering object geometry and reflectance under complex environmental illumination, a limitation of existing inverse rendering methods. To overcome this, we propose a three-stage inverse rendering framework that fuses multi-view RGB images with active near-infrared (NIR) flash illumination. Leveraging the insensitivity of NIR to ambient lighting, our approach effectively eliminates illumination interference while exploiting the complementary strengths of RGB and NIR modalities. We introduce, for the first time, invisible NIR flash imaging into inverse rendering and present the first multi-view RGB-NIR inverse rendering dataset. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches across diverse lighting conditions, achieving more stable and accurate geometry and material reconstruction.

Abstract

Inverse rendering aims to reconstruct geometry and reflectance of objects from images. Despite recent progress, existing methods often produces inaccurate reconstructions that are sensitive to ambient illumination conditions. Here we introduce an ambient-robust inverse rendering method enabled by active RGB-NIR imaging. Our key insight is to leverage near-infrared (NIR) flash illumination-imperceptible to human observers-to obtain stable point-light shading that is largely invariant to ambient illumination. By using multi-view RGB images illuminated by ambient light and NIR images acquired with active NIR flash illumination, we reconstruct accurate geometry and reflectance by exploiting the complementary benefits of RGB and NIR images via a three-stage inverse rendering method. To enable dense multi-view acquisition, we develop an active imaging system equipped with a RGB-NIR camera and a NIR flash mounted on a mobile base. Using this system, we collect the first multi-view RGB-NIR inverse rendering dataset captured under multiple ambient illumination conditions. Experiments demonstrate that our method outperforms prior approaches, achieving accurate geometry and reflectance estimation across multiple ambient lighting scenarios.

Problem

• inverse rendering
• ambient illumination
• geometry reconstruction
• reflectance estimation
• RGB-NIR imaging

Innovation

• inverse rendering
• active imaging
• RGB-NIR
• ambient-robust
• near-infrared