RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting

2026-09-07

AI Summary

为解决图像重光照问题，提出RelightFormer模型，采用前馈生成式Transformer处理单视图和多视图图像，通过动态注入环境图和无序多视图输入对称处理来实现。

Abstract

Image relighting is traditionally tackled via complex inverse rendering pipelines, which suffer from ill-posed optimization, or single-image generative models that ignore crucial multi-view cues necessary for understanding 3D geometry and material interactions. To address these limitations, we introduce a feed-forward generative Transformer for direct single- and multi-view image relighting that entirely bypasses explicit intrinsic property estimation. Adapted from a video foundation model, our architecture features a latent illumination module that dynamically injects target environment maps into spatial features via cross-attention. Furthermore, we employ permutation-invariant positional encodings to symmetrically process unordered multi-view inputs without sequential bias. To train this robust data-driven model, we construct the massive Laval Objaverse Dataset (LOD), comprising 90K objects and 39K unique illuminations. Extensive experiments demonstrate state-of-the-art visual quality, photorealistic relighting quality, and strong zero-shot generalization across single-view, multi-view, and novel-view relighting tasks.

Problem

• Image relighting
• inverse rendering
• multi-view cues

Innovation

• feed-forward generative Transformer
• latent illumination module
• permutation-invariant positional encodings
• Laval Objaverse Dataset