Understanding LoRA Rank Trade-offs in Diffusion Model Fine-Tuning

2026-09-09

AI Summary

研究通过控制实验分析了不同LoRA秩在扩散模型微调中的性能与成本平衡，结果表明中等秩（如4或8）在保持高质量的同时具有最高的效率。

Abstract

Selecting LoRA rank for diffusion fine-tuning requires balancing quality and compute cost. We present a controlled study on CIFAR-10 using a DDPM U-Net with ranks {2,4,8,16,32}, fixed optimization settings, and a reproducible local-folder pytorch-fid protocol. We report FID, trainable parameters, runtime, and GPU memory, then validate trends with extended-budget DDPM runs (20 epochs; ranks 4/8/16) and a Tiny DiT backbone (10 epochs; ranks 4/8/16). Results show moderate ranks are most efficient: rank 4 achieves the best DDPM FID (124.1380), rank 8 is close (124.2136), and higher ranks provide limited gains despite larger adaptation cost. These findings support small-to-moderate ranks as practical defaults under fixed training budgets.

Problem

• LoRA rank
• diffusion model
• fine-tuning
• quality
• compute cost

Innovation

• LoRA Rank
• Diffusion Model Fine-Tuning
• FID Score
• Efficiency Trade-off
• Controlled Study