TransLight: Image-Guided Customized Lighting Control with Generative Decoupling

2025-08-20

AI Summary

Existing illumination editing methods struggle to simultaneously achieve customizable lighting control and content fidelity, particularly in cross-image complex illumination transfer tasks. To address this, we propose a generative disentanglement framework that, for the first time, achieves complete separation of content and illumination features in real-world scenes. We construct a million-scale image–content–illumination triplet dataset and perform end-to-end training on a fine-tuned diffusion model integrated with the IC-Light architecture, conditioned on reference illumination maps. Our approach enables high-fidelity, highly flexible illumination transfer, significantly improving cross-domain illumination harmony and editing naturalness. Quantitative and qualitative evaluations demonstrate that our method surpasses state-of-the-art approaches across multiple metrics—including LPIPS, SSIM, and user studies—while enabling unprecedented control over lighting attributes without compromising structural or textural integrity. This work establishes a new paradigm for illumination editing and harmonization, advancing both theoretical understanding and practical applicability in photorealistic image synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Most existing illumination-editing approaches fail to simultaneously provide customized control of light effects and preserve content integrity. This makes them less effective for practical lighting stylization requirements, especially in the challenging task of transferring complex light effects from a reference image to a user-specified target image. To address this problem, we propose TransLight, a novel framework that enables high-fidelity and high-freedom transfer of light effects. Extracting the light effect from the reference image is the most critical and challenging step in our method. The difficulty lies in the complex geometric structure features embedded in light effects that are highly coupled with content in real-world scenarios. To achieve this, we first present Generative Decoupling, where two fine-tuned diffusion models are used to accurately separate image content and light effects, generating a newly curated, million-scale dataset of image-content-light triplets. Then, we employ IC-Light as the generative model and train our model with our triplets, injecting the reference lighting image as an additional conditioning signal. The resulting TransLight model enables customized and natural transfer of diverse light effects. Notably, by thoroughly disentangling light effects from reference images, our generative decoupling strategy endows TransLight with highly flexible illumination control. Experimental results establish TransLight as the first method to successfully transfer light effects across disparate images, delivering more customized illumination control than existing techniques and charting new directions for research in illumination harmonization and editing.

Problem

• Transferring complex light effects between images
• Decoupling light effects from image content
• Providing customized illumination control while preserving integrity

Innovation

• Generative Decoupling separates content and light effects
• Uses two fine-tuned diffusion models for accurate separation
• IC-Light model trained with million-scale image-content-light triplets