🤖 AI Summary
Virtual try-on (VTON) faces challenges including clothing detail loss, inaccurate human-clothing alignment, inefficient inference, and poor generalization across poses and styles. This paper proposes DiffFit, a two-stage latent diffusion framework that— for the first time—decouples geometry-aware clothing deformation from texture fidelity optimization. In Stage I, a fine-grained deformation network achieves pose-adaptive geometric alignment; in Stage II, a cross-modal conditional diffusion model synthesizes high-fidelity appearance by jointly conditioning on the original garment texture, target person image, and deformed intermediate result. This progressive design significantly improves generation stability, detail preservation, and cross-pose generalization. Evaluated on large-scale benchmarks, DiffFit surpasses state-of-the-art methods in FID, LPIPS, and user studies, achieving simultaneous optimization of inference efficiency and visual quality.
📝 Abstract
Virtual try-on (VTON) aims to synthesize realistic images of a person wearing a target garment, with broad applications in e-commerce and digital fashion. While recent advances in latent diffusion models have substantially improved visual quality, existing approaches still struggle with preserving fine-grained garment details, achieving precise garment-body alignment, maintaining inference efficiency, and generalizing to diverse poses and clothing styles. To address these challenges, we propose DiffFit, a novel two-stage latent diffusion framework for high-fidelity virtual try-on. DiffFit adopts a progressive generation strategy: the first stage performs geometry-aware garment warping, aligning the garment with the target body through fine-grained deformation and pose adaptation. The second stage refines texture fidelity via a cross-modal conditional diffusion model that integrates the warped garment, the original garment appearance, and the target person image for high-quality rendering. By decoupling geometric alignment and appearance refinement, DiffFit effectively reduces task complexity and enhances both generation stability and visual realism. It excels in preserving garment-specific attributes such as textures, wrinkles, and lighting, while ensuring accurate alignment with the human body. Extensive experiments on large-scale VTON benchmarks demonstrate that DiffFit achieves superior performance over existing state-of-the-art methods in both quantitative metrics and perceptual evaluations.