PanDORA: Casual HDR Radiance Acquisition for Indoor Scenes

2024-07-08

arXiv.org

AI Summary

Existing NeRF methods, constrained by LDR input, struggle to reconstruct realistic HDR indoor radiance fields; exposure bracketing alleviates this but is time-consuming and impractical. This paper introduces the first portable dual-exposure 360° cooperative capture system, enabling users to freely swing the device for several minutes to acquire HDR data via synchronized acquisition of conventional and ultra-short-exposure omnidirectional video. Our method integrates dual-camera hardware, a NeRF-based HDR radiance reconstruction framework, and joint multi-exposure fusion with inverse rendering modeling. It achieves high-fidelity indoor HDR NeRF reconstruction without exposure bracketing or precise motion control—marking the first such solution. Experiments demonstrate full recovery of specular highlights from direct light sources (e.g., LED fixtures, windows), significantly broader dynamic range than state-of-the-art HDR baselines, and NeRF-level visual fidelity.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Most novel view synthesis methods such as NeRF are unable to capture the true high dynamic range (HDR) radiance of scenes since they are typically trained on photos captured with standard low dynamic range (LDR) cameras. While the traditional exposure bracketing approach which captures several images at different exposures has recently been adapted to the multi-view case, we find such methods to fall short of capturing the full dynamic range of indoor scenes, which includes very bright light sources. In this paper, we present PanDORA: a PANoramic Dual-Observer Radiance Acquisition system for the casual capture of indoor scenes in high dynamic range. Our proposed system comprises two 360{deg} cameras rigidly attached to a portable tripod. The cameras simultaneously acquire two 360{deg} videos: one at a regular exposure and the other at a very fast exposure, allowing a user to simply wave the apparatus casually around the scene in a matter of minutes. The resulting images are fed to a NeRF-based algorithm that reconstructs the scene's full high dynamic range. Compared to HDR baselines from previous work, our approach reconstructs the full HDR radiance of indoor scenes without sacrificing the visual quality while retaining the ease of capture from recent NeRF-like approaches.

Problem

• Capturing true HDR radiance in indoor scenes using conventional cameras
• Reducing acquisition time compared to exposure bracketing techniques
• Generating non-saturated HDR radiance maps from panoramic video data

Innovation

• Uses two 360° cameras on monopod for HDR capture
• Processes videos with two-stage NeRF-based alignment algorithm
• Generates non-saturated HDR radiance maps from dual exposures