Decoupled Illumination Priors for Spatially Controllable Multi-View Indoor Scene Relighting

2026-07-09

AI Summary

Existing diffusion models struggle to simultaneously achieve photorealism, spatial lighting control accuracy, and multi-view consistency in indoor scene relighting, often compromising generative priors when specifying 3D light source positions. To address this, we propose Lume-Palette, a framework that decouples relighting into two stages: illumination distillation and illumination projection. The first stage extracts a “lighting palette” from a pre-trained diffusion model, preserving realistic material–illumination interactions, while the second stage explicitly maps target spatial illumination onto a coarse 3D geometry. By disentangling semantic lighting priors from spatial control and incorporating an asymmetric multi-view conditioning strategy, our method achieves high-fidelity, spatially accurate, and multi-view consistent relighting results on both synthetic and real-world indoor scenes.

Abstract

Indoor scene relighting demands photorealism, precise spatial control, and strict multi-view consistency. While diffusion-based image editing models enable semantic lighting manipulation via text prompts, enforcing exact 3D light placement often disrupts their generative priors. We propose Lume-Palette, a progressive framework that leverages semantic lighting priors for spatially controllable multi-view indoor relighting. The approach decouples relighting into two stages: (1) illumination distillation, which extracts canonical illumination palettes from a pretrained diffusion model to preserve realistic material-light interactions, and (2) illumination casting, which explicitly maps target spatial lighting conditions defined from coarse 3D geometry. To efficiently handle dense multi-view and multi-modal inputs, we introduce an asymmetric multi-view conditioning strategy that selectively injects essential spatial context. Experiments on diverse synthetic scenes and real-world scenes demonstrate that Lume-Palette produces photorealistic, spatially controllable, and multi-view consistent relighting results. Project Page: https://cjeen.github.io/lumepalette

Problem

• indoor scene relighting
• spatially controllable lighting
• multi-view consistency
• photorealism
• 3D light placement

Innovation

• decoupled illumination
• spatially controllable relighting
• multi-view consistency
• diffusion priors
• asymmetric multi-view conditioning