PPISP: Physically-Plausible Compensation and Control of Photometric Variations in Radiance Field Reconstruction

2026-01-26

AI Summary

This work addresses the performance degradation in multi-view 3D reconstruction caused by photometric inconsistencies arising from camera optical characteristics and image signal processing (ISP). Existing approaches often lack physical grounding and exhibit limited generalization. To overcome these limitations, we propose the first physically interpretable ISP correction module that disentangles camera-intrinsic and capture-dependent photometric effects. We introduce a PPISP controller to predict ISP parameters for novel views, enabling realistic and photometrically consistent novel view synthesis. Our method integrates physics-based photometric modeling, interpretable ISP parameterization, and neural radiance fields, supporting exposure- and white-balance-like control mechanisms and principled evaluation—all without requiring ground-truth images. Experiments on standard benchmarks demonstrate state-of-the-art performance, along with intuitive photometric control and seamless metadata integration.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Security and Privacy: Data transparency and provenance

Abstract

Multi-view 3D reconstruction methods remain highly sensitive to photometric inconsistencies arising from camera optical characteristics and variations in image signal processing (ISP). Existing mitigation strategies such as per-frame latent variables or affine color corrections lack physical grounding and generalize poorly to novel views. We propose the Physically-Plausible ISP (PPISP) correction module, which disentangles camera-intrinsic and capture-dependent effects through physically based and interpretable transformations. A dedicated PPISP controller, trained on the input views, predicts ISP parameters for novel viewpoints, analogous to auto exposure and auto white balance in real cameras. This design enables realistic and fair evaluation on novel views without access to ground-truth images. PPISP achieves SoTA performance on standard benchmarks, while providing intuitive control and supporting the integration of metadata when available. The source code is available at: https://github.com/nv-tlabs/ppisp

Problem

• photometric inconsistency
• multi-view 3D reconstruction
• image signal processing
• novel view synthesis
• camera optics

Innovation

• physically-plausible
• ISP correction
• radiance field reconstruction
• photometric consistency
• novel view synthesis