HorizonRelight: Relighting Long-horizon Videos Consistently via Diffusion Transformers

2026-06-27

AI Summary

This work addresses temporal discontinuities in long-duration video relighting using diffusion models, which commonly arise from patch-based sliding-window inference and manifest as artifacts and abrupt appearance changes at block boundaries. To mitigate this issue, the task is reformulated as a temporally conditioned latent-space translation problem. The authors propose a masked target-domain self-conditioning mechanism that propagates target latent variables across blocks to enforce temporal consistency, along with a warm-start prompting strategy that enables prompt-guided controllable relighting. Integrated within a diffusion Transformer architecture and combined with an optimized sliding-window inference scheme, the method significantly reduces boundary artifacts and effectively preserves cross-frame appearance coherence on real-world long videos.

Abstract

Diffusion-based video relighting enables controllable relighting from a single input video, but modern video diffusion backbones are trained on short clips and applied to long-horizon videos through chunked sliding-window inference, often causing temporal discontinuities at chunk boundaries. We address this by reframing long-horizon relighting as \emph{temporally conditioned latent domain translation}. Our framework enforces cross-chunk continuity by propagating target-domain latents across boundaries and makes this behavior learnable using \emph{masked target-domain self-conditioning}, training the model to continue from temporally masked propagated context. We further introduce \emph{warm-start prompting} with a relit prompt anchor from a controllable generative model, which establishes the initial target-domain state and creates a general interface for prompt-based relighting. Experiments on in-the-wild long-horizon videos show markedly improved temporal consistency, with chunk-boundary artifacts largely reduced and unwanted appearance changes across chunks greatly suppressed.

Problem

• video relighting
• long-horizon videos
• temporal consistency
• chunk boundaries
• diffusion models

Innovation

• temporally conditioned latent domain translation
• masked target-domain self-conditioning
• warm-start prompting
• diffusion transformers
• long-horizon video relighting