PI-Light: Physics-Inspired Diffusion for Full-Image Relighting

2026-01-29

AI Summary

This work addresses the challenges of full-image relighting, including the scarcity of paired data, difficulties in ensuring physical plausibility, and limited generalization of data-driven priors. The authors propose a two-stage physics-inspired diffusion framework that enhances intrinsic attribute consistency through batch-aware attention, integrates physics-guided neural rendering, and employs physically constrained loss functions to guarantee physically accurate light transport. To facilitate effective fine-tuning, they also construct a high-quality dataset of controllable illumination images. The method accurately synthesizes both specular highlights and diffuse reflections across diverse materials, significantly outperforming existing approaches and demonstrating exceptional generalization capabilities in real-world relighting tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Full-image relighting remains a challenging problem due to the difficulty of collecting large-scale structured paired data, the difficulty of maintaining physical plausibility, and the limited generalizability imposed by data-driven priors. Existing attempts to bridge the synthetic-to-real gap for full-scene relighting remain suboptimal. To tackle these challenges, we introduce Physics-Inspired diffusion for full-image reLight ($\pi$-Light, or PI-Light), a two-stage framework that leverages physics-inspired diffusion models. Our design incorporates (i) batch-aware attention, which improves the consistency of intrinsic predictions across a collection of images, (ii) a physics-guided neural rendering module that enforces physically plausible light transport, (iii) physics-inspired losses that regularize training dynamics toward a physically meaningful landscape, thereby enhancing generalizability to real-world image editing, and (iv) a carefully curated dataset of diverse objects and scenes captured under controlled lighting conditions. Together, these components enable efficient finetuning of pretrained diffusion models while also providing a solid benchmark for downstream evaluation. Experiments demonstrate that $\pi$-Light synthesizes specular highlights and diffuse reflections across a wide variety of materials, achieving superior generalization to real-world scenes compared with prior approaches.

Problem

• full-image relighting
• physical plausibility
• generalizability
• synthetic-to-real gap
• paired data

Innovation

• physics-inspired diffusion
• full-image relighting
• neural rendering
• batch-aware attention
• physically plausible lighting