WarpI2I: Image Warping for Image-to-Image Translation

2026-06-29

AI Summary

This work addresses the loss of fine-grained structural details in compact latent diffusion models for image-to-image translation, which arises from spatial downsampling during encoding. To mitigate this, the authors propose a saliency-guided image warping–unwarping framework that dynamically reallocates spatial resolution to salient regions prior to encoding, processes the data through a standard diffusion model, and then inversely maps the output back to the original spatial domain. This approach enhances structural detail preservation without increasing latent resolution or altering the model architecture. Coupled with an outpainting-based synthetic data generation strategy, it enables efficient, model-agnostic construction of high-quality paired data with improved structural fidelity. Experiments demonstrate significant gains in structural accuracy, illumination correctness, and overall image quality on portrait and driving-scene relighting and translation tasks, along with strong temporal consistency in video applications.

Abstract

Image-to-image (I2I) translation has achieved strong results in tasks like human relighting and driving scene translation using latent diffusion models (LDMs). However, compact LDMs often struggle to preserve fine-grained structures because the encoder compresses high-resolution inputs into a spatially downsampled latent space. To address this issue, we propose a simple saliency-guided warp-unwarp framework that reallocates spatial representation toward salient regions before encoding, enabling better preservation of structural details without increasing latent resolution. The warped image is processed by the original diffusion model and then mapped back via an inverse warp. In addition, we propose a simple and efficient outpainting-based synthetic data generation pipeline to produce high-quality paired data for image relighting. Our method is model-agnostic, requires no architectural modification, and introduces negligible computational overhead. Experiments on human relighting, driving scene relighting, and translation demonstrate improved structural preservation, lighting faithfulness, and image quality, with our framework extending naturally to video via frame-by-frame application with good temporal stability. Project Webpage: https://shenzheng2000.github.io/WarpI2I.github.io

Problem

• image-to-image translation
• latent diffusion models
• structural preservation
• spatial downsampling
• fine-grained structures

Innovation

• saliency-guided warping
• latent diffusion models
• image-to-image translation
• structural preservation
• outpainting-based data synthesis