FreeDNA: Endowing Domain Adaptation of Diffusion-Based Dense Prediction with Training-Free Domain Noise Alignment

2025-06-25

AI Summary

This work addresses the training-free domain adaptation problem for diffusion models in dense prediction tasks. We propose Domain Noise Alignment (DNA), a fine-tuning-free method that dynamically calibrates the noise statistical distributions between source and target domains during sampling. Specifically, DNA estimates and adjusts the target-domain noise variance using high-confidence regions to achieve cross-domain noise distribution matching. To our knowledge, this is the first approach to integrate domain adaptation into diffusion-based dense prediction frameworks, supporting both source-available and source-free scenarios within a unified formulation. We validate DNA on four canonical dense prediction tasks—semantic segmentation, depth estimation, surface normal prediction, and edge detection—demonstrating consistent and significant improvements in cross-domain generalization performance. The implementation is publicly available.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Application Domains: Transportation

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Domain Adaptation(DA) for dense prediction tasks is an important topic, which enhances the dense prediction model's performance when tested on its unseen domain. Recently, with the development of Diffusion-based Dense Prediction (DDP) models, the exploration of DA designs tailored to this framework is worth exploring, since the diffusion model is effective in modeling the distribution transformation that comprises domain information. In this work, we propose a training-free mechanism for DDP frameworks, endowing them with DA capabilities. Our motivation arises from the observation that the exposure bias (e.g., noise statistics bias) in diffusion brings domain shift, and different domains in conditions of DDP models can also be effectively captured by the noise prediction statistics. Based on this, we propose a training-free Domain Noise Alignment (DNA) approach, which alleviates the variations of noise statistics to domain changes during the diffusion sampling process, thereby achieving domain adaptation. Specifically, when the source domain is available, we directly adopt the DNA method to achieve domain adaptation by aligning the noise statistics of the target domain with those of the source domain. For the more challenging source-free DA, inspired by the observation that regions closer to the source domain exhibit higher confidence meeting variations of sampling noise, we utilize the statistics from the high-confidence regions progressively to guide the noise statistic adjustment during the sampling process. Notably, our method demonstrates the effectiveness of enhancing the DA capability of DDP models across four common dense prediction tasks. Code is available at href{https://github.com/xuhang07/FreeDNA}{https://github.com/xuhang07/FreeDNA}.

Problem

• Enhancing domain adaptation for diffusion-based dense prediction models
• Aligning noise statistics to reduce domain shift without training
• Improving performance across multiple dense prediction tasks

Innovation

• Training-free Domain Noise Alignment method
• Aligns target noise with source statistics
• Uses high-confidence regions for source-free DA