Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learning

2025-02-06

AI Summary

Conditional diffusion models suffer from low sample efficiency in few-shot transfer learning. Method: We propose the first provably efficient representation transfer framework, which learns a shared low-dimensional conditional representation on the source task to substantially improve sample efficiency on the target task. Contribution/Results: Theoretically, we establish the first sample complexity analysis for transfer learning of conditional diffusion models, rigorously proving that the target-task sample complexity decreases polynomially with the dimensionality of the shared representation and providing verifiable theoretical guarantees. Methodologically, our approach jointly optimizes representation learning and conditional diffusion modeling to enable cross-task conditional generation transfer. Empirically, we validate our framework on diverse real-world conditional generation tasks—including class-conditional image synthesis and text-to-image generation—demonstrating consistent improvements in transfer performance that align closely with theoretical predictions. Our results confirm that low-dimensional conditional representations are a critical factor for enhancing few-shot generalization in diffusion-based generative modeling.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

While conditional diffusion models have achieved remarkable success in various applications, they require abundant data to train from scratch, which is often infeasible in practice. To address this issue, transfer learning has emerged as an essential paradigm in small data regimes. Despite its empirical success, the theoretical underpinnings of transfer learning conditional diffusion models remain unexplored. In this paper, we take the first step towards understanding the sample efficiency of transfer learning conditional diffusion models through the lens of representation learning. Inspired by practical training procedures, we assume that there exists a low-dimensional representation of conditions shared across all tasks. Our analysis shows that with a well-learned representation from source tasks, the samplecomplexity of target tasks can be reduced substantially. In addition, we investigate the practical implications of our theoretical results in several real-world applications of conditional diffusion models. Numerical experiments are also conducted to verify our results.

Problem

• Sample-efficient transfer learning
• Conditional diffusion models
• Representation learning

Innovation

• Transfer learning conditional diffusion models
• Low-dimensional representation of conditions
• Sample complexity reduction via representation