Lighting-Consistent Object Transfer Across Radiance Fields

2026-06-21

AI Summary

This work addresses the challenge of photometric inconsistency when transferring objects across scenes using 3D Gaussian Splatting (3DGS), which often leads to unrealistic composites due to mismatched illumination. To resolve this, the authors propose the first end-to-end 3D method that integrates a diffusion model into the 3DGS framework. The approach first renders views containing the target object under inconsistent lighting, then employs a diffusion model to harmonize the illumination, and finally refines the result through post-optimization to produce a coherent 3D Gaussian representation. By leveraging a heterogeneous training set composed of synthetic, generative, and real-world data, the method significantly outperforms existing techniques, achieving high visual fidelity and illumination consistency in cross-scene object composition and thereby enhancing the practicality of 3DGS for content editing.

Abstract

3D Gaussian Splatting (3DGS) is widely used to capture and render real scenes. Compositing objects from one capture into another has applications in many domains, such as VFX, architecture and interior design, or marketing. However, extracting an object from a source scene and naively pasting it into a target scene will fail to produce realistic results due to the different lighting conditions between the two scenes. To address this problem, we introduce a diffusion model that harmonizes naively composited images with inconsistent lighting. The model is trained with a heterogeneous dataset of image pairs (inconsistent composite input, consistent output), combining synthetic, generated, and real data. Our complete 3D solution allows a user to extract an object from the source scene and composite it into the target scene. From this, the (inconsistent) views of the target scene with the composite object are rendered. Our diffusion model harmonizes each one of these views, which are finally consolidated in a 3DGS representation with a post-optimization step. Our method provides visually compelling results, making object transfer between 3DGS easy to use and significantly improving quality compared to previous methods.

Problem

• lighting consistency
• object transfer
• 3D Gaussian Splatting
• radiance fields
• image harmonization

Innovation

• 3D Gaussian Splatting
• lighting harmonization
• diffusion model
• object transfer
• radiance fields