ResEdit: Residual embeddings for precise generative image editing

2026-06-15

AI Summary

This work addresses the challenge of simultaneously preserving identity consistency and structural stability in generative image editing when no paired fine-tuning data is available. To this end, the authors propose a decoupled editing framework based on residual image encoding and gradient reversal optimization. By incorporating residual embeddings as an additional conditioning signal in the diffusion model, the method enhances reconstruction fidelity and reduces reliance on weakly conditioned inversion. A gradient reversal layer is further employed to effectively disentangle the edited attributes from the original content. The approach demonstrates significant improvements in editing fidelity and consistency across tasks such as relighting and text-guided manipulation, surpassing the performance limitations of existing training-free image editing methods.

Abstract

Conditional diffusion image generators can be repurposed for editing through inversion, without the need for large-scale paired fine-tuning data. However, producing high-quality, targeted edits while maintaining image identity and global consistency remains challenging, as weakly conditioned inversion often embeds conflicting image features into the noise. We demonstrate that incorporating a residual image encoding as additional conditioning enables both improved identity preservation and better editability. We optimize this residual encoding to provide a strong conditioning signal for reconstruction, thereby reducing the reliance on inversion and susceptibility to its aforementioned pitfalls. To ensure this residual does not interfere with desired edits, we incorporate a gradient reversal-based optimization strategy that disentangles the residual from the edited condition. We illustrate our method's ability to produce high-fidelity results across precise intrinsic-based editing and relighting, and show proof-of-concept text-guided manipulation.

Problem

• image editing
• identity preservation
• diffusion models
• conditional generation
• global consistency

Innovation

• residual embeddings
• diffusion models
• image editing
• gradient reversal
• identity preservation