FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement

2025-11-23

AI Summary

Video relighting and background replacement suffer from temporal inconsistency, spatial distortion, and unnatural illumination. To address these challenges, this paper proposes a training-free, end-to-end video editing framework. First, a residual-corrected optical flow mechanism is introduced to ensure lossless reconstruction under input consistency while preserving temporal coherence. Second, a decoupled conditional control scheme with foreground masking is employed to separately govern foreground relighting and background generation. Third, high-frequency information transfer is integrated to enhance detail fidelity and illumination realism. The method achieves significant improvements in temporal consistency, structural fidelity, and illumination naturalness—without requiring fine-tuning—while maintaining high computational efficiency. It generalizes robustly across diverse scenes and lighting conditions.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Multimodal Learning
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Video relighting with background replacement is a challenging task critical for applications in film production and creative media. Existing methods struggle to balance temporal consistency, spatial fidelity, and illumination naturalness. To address these issues, we introduce FlowPortal, a novel training-free flow-based video relighting framework. Our core innovation is a Residual-Corrected Flow mechanism that transforms a standard flow-based model into an editing model, guaranteeing perfect reconstruction when input conditions are identical and enabling faithful relighting when they differ, resulting in high structural consistency. This is further enhanced by a Decoupled Condition Design for precise lighting control and a High-Frequency Transfer mechanism for detail preservation. Additionally, a masking strategy isolates foreground relighting from background pure generation process. Experiments demonstrate that FlowPortal achieves superior performance in temporal coherence, structural preservation, and lighting realism, while maintaining high efficiency. Project Page: https://gaowenshuo.github.io/FlowPortalProject/.

Problem

• Achieving temporal consistency in video relighting with background replacement
• Balancing spatial fidelity and illumination naturalness in video editing
• Enabling training-free video relighting while preserving structural details

Innovation

• Residual-Corrected Flow mechanism enables training-free video editing
• Decoupled Condition Design provides precise lighting control
• High-Frequency Transfer mechanism preserves structural details