TokenLight: Precise Lighting Control in Images using Attribute Tokens

2026-04-16

AI Summary

This work proposes an attribute-token-based method for image relighting, formulating illumination editing as a conditional image generation task. By introducing learnable attribute tokens, the approach disentangles and implicitly encodes multidimensional lighting parameters—including intensity, color, ambient light, diffuse components, and 3D light source positions—without requiring inverse rendering supervision. This enables implicit modeling of the complex interactions among lighting, geometry, material properties, and occlusion. Leveraging a diffusion-based generative architecture, large-scale synthetic data pretraining, and fine-tuning on real images, the method achieves state-of-the-art performance on both synthetic and real-world datasets. It demonstrates exceptional realism and controllability in challenging scenarios such as placing light sources inside objects or relighting transparent materials.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This paper presents a method for image relighting that enables precise and continuous control over multiple illumination attributes in a photograph. We formulate relighting as a conditional image generation task and introduce attribute tokens to encode distinct lighting factors such as intensity, color, ambient illumination, diffuse level, and 3D light positions. The model is trained on a large-scale synthetic dataset with ground-truth lighting annotations, supplemented by a small set of real captures to enhance realism and generalization. We validate our approach across a variety of relighting tasks, including controlling in-scene lighting fixtures and editing environment illumination using virtual light sources, on synthetic and real images. Our method achieves state-of-the-art quantitative and qualitative performance compared to prior work. Remarkably, without explicit inverse rendering supervision, the model exhibits an inherent understanding of how light interacts with scene geometry, occlusion, and materials, yielding convincing lighting effects even in traditionally challenging scenarios such as placing lights within objects or relighting transparent materials plausibly. Project page: vrroom.github.io/tokenlight/

Problem

• image relighting
• lighting control
• attribute tokens
• illumination editing
• conditional image generation

Innovation

• attribute tokens
• image relighting
• conditional image generation
• lighting control
• inverse rendering-free