ToPO: Token-Conditioned Preference Routing for Attention-Based Latent Diffusion Models

2026-09-03

AI Summary

为了解决Diffusion-DPO在处理偏好标签时的效率问题，ToPO通过构建空间-时间路由和使用内容令牌调节来优化基于注意力的潜在扩散模型。

Abstract

Pairwise preference labels rank complete images, yet Diffusion-DPO applies their effect over many spatial and denoising-time coordinates. For attention-based, noise-prediction latent diffusion, ToPO (Token-Oriented Preference Optimization) constructs a per-minibatch, detached, separable spatial-temporal route from branchwise squared-residual contrast in a frozen reference denoiser. Preferred-branch cross-attention uses content tokens to modulate the spatial factor, and an auxiliary pixel-midpoint ordering term is added without local labels or a learned reward model. In matched three-seed retrainings with a shared update schedule, ToPO has higher endpoint estimates than Diffusion-DPO on all five reported SD-1.5 metrics and on HPSv2, ImageReward, and CLIP for SDXL. It also receives larger raw win shares in an aggregate blind SDXL A/B study. These findings are scoped to the reported equal-update U-Net protocols rather than an equal-compute comparison.

Problem

• Pairwise preference labels
• Attention-based latent diffusion
• Spatial-temporal route

Innovation

• Token-Oriented Preference Optimization
• attention-based latent diffusion models
• spatial-temporal route
• branchwise squared-residual contrast
• cross-attention