CORAL: Correspondence Alignment for Improved Virtual Try-On

2026-02-19

AI Summary

Existing unpaired virtual try-on methods struggle to preserve fine garment details and lack explicit mechanisms for aligning person–garment correspondences. This work reveals, for the first time, that correspondence modeling in Diffusion Transformers (DiTs) hinges on precise query–key matching, and proposes an explicit alignment strategy that combines correspondence distillation loss with attention entropy minimization to steer attention toward reliable external correspondence points. Built upon the DiT architecture, the proposed method outperforms current baselines in both global shape transfer and local detail preservation. Ablation studies confirm the effectiveness of each component, and a novel evaluation protocol based on vision–language models is introduced to provide a more comprehensive assessment of generation quality.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Existing methods for Virtual Try-On (VTON) often struggle to preserve fine garment details, especially in unpaired settings where accurate person-garment correspondence is required. These methods do not explicitly enforce person-garment alignment and fail to explain how correspondence emerges within Diffusion Transformers (DiTs). In this paper, we first analyze full 3D attention in DiT-based architecture and reveal that the person-garment correspondence critically depends on precise person-garment query-key matching within the full 3D attention. Building on this insight, we then introduce CORrespondence ALignment (CORAL), a DiT-based framework that explicitly aligns query-key matching with robust external correspondences. CORAL integrates two complementary components: a correspondence distillation loss that aligns reliable matches with person-garment attention, and an entropy minimization loss that sharpens the attention distribution. We further propose a VLM-based evaluation protocol to better reflect human preference. CORAL consistently improves over the baseline, enhancing both global shape transfer and local detail preservation. Extensive ablations validate our design choices.

Problem

• Virtual Try-On
• person-garment correspondence
• Diffusion Transformers
• unpaired setting
• garment detail preservation

Innovation

• Correspondence Alignment
• Diffusion Transformer
• Virtual Try-On
• Attention Mechanism
• Correspondence Distillation