LumiTokens: 3D Relighting via Token-Space Lighting Transformation

2026-08-18

AI Summary

本文提出LumiTokens，通过在潜在场景令牌上直接进行变换来实现3D重新照明，无需显式3D表示或物理分解，支持渐进和可组合的照明编辑。

Abstract

Existing 3D relighting methods operate through either explicit material decomposition, diffusion-based view-space generation, or a combination of both, requiring full recomputation for each new lighting condition. We observe that recent latent scene representations, which encode multi-view images into a set of compact tokens with no fixed physical semantics, open up a novel design space for relighting. We present LumiTokens, a framework that formulates 3D relighting as a direct transformation on latent scene tokens, without explicit 3D representations, rendering equations, or physics-based decomposition. Our model introduces a Scene Token Editor that processes scene tokens jointly with light-ray tokens through self-attention, producing updated tokens that can be decoded into multi-view-consistent relit images. To support diverse lighting types through a unified interface, all lighting signals, including environment maps, point lights, and area lights, are parameterized as Plucker ray tokens, enabling native 3D user interaction with a representation that carries no explicit spatial structure. Crucially, this design supports progressive relighting: because the editor's output remains in the same latent space as its input, a user can incrementally build up illumination one light source at a time, with each edit composing in token space. Experiments demonstrate that LumiTokens achieves comparable or superior relighting quality to other methods and supports progressive, composable lighting edits. Project page: https://neu-vi.github.io/LumiTokens/

Problem

• 3D Relighting
• Latent Scene Representation
• Lighting Transformation
• Token-Space

Innovation

• Token-Space Lighting Transformation
• Scene Token Editor
• Plucker ray tokens
• Progressive relighting