Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models

2026-05-28

AI Summary

Diffusion models in text-to-image generation often suffer from counting errors and detail inconsistencies due to insufficient semantic alignment. This work proposes a lightweight, reward-free post-training method that, for the first time, directly incorporates contrastive alignment guidance into the score-matching objective of diffusion models, optimizing soft textual tokens at the score level to enhance semantic consistency. By circumventing the excessive penalization of negative samples inherent in conventional contrastive learning, the approach effectively mitigates issues of repetition and overcounting. It is compatible with mainstream architectures including Stable Diffusion 1.5, SDXL, and SD3, achieving a counting accuracy improvement of over 35% on the GenEval benchmark—matching the performance of SoftREPA—and demonstrates complementary benefits when combined with reinforcement learning methods.

Abstract

Diffusion models generate highly realistic images but often struggle with precise text-image alignment. While recent post-training methods improve alignment using external rewards or human preference signals, their performance heavily depends on reward quality and does not directly address alignment within the diffusion process itself. Recent reward-free approaches such as SoftREPA demonstrate that optimizing soft text tokens via contrastive learning can effectively improve text-image representation alignment, outperforming standard parameter-efficient fine-tuning baselines. However, the contrastive formulation can excessively penalize negative pairs, which manifests as characteristic failure cases such as over-counting and repetition. To address this issue, we propose a lightweight, reward-free post-training method that refines soft tokens by integrating contrastive alignment guidance directly into the score-matching objective of diffusion models. By assigning alignment directions at the score level, our approach mitigates these limitations and yields more coherent and semantically faithful generations. Experiments show that our method matches SoftREPA while substantially improving its failure cases, achieving over 35% improvement in counting accuracy on the GenEval benchmark. Our method is seamlessly applicable to existing diffusion backbones (SD1.5, SDXL, and SD3), and is complementary to existing RL-based diffusion post-training methods. Project page: https://jaayeon.github.io/AGSM

Problem

• text-to-image alignment
• diffusion models
• contrastive learning
• score matching
• post-training

Innovation

• score matching
• text-to-image alignment
• contrastive learning
• diffusion models
• reward-free fine-tuning