Edge-Assisted Collaborative Fine-Tuning for Multi-User Personalized Artificial Intelligence Generated Content (AIGC)

2025-08-06

AI Summary

To address the challenges of resource constraints, privacy sensitivity, and multi-user personalized AIGC support on edge devices, this paper proposes a cluster-aware hierarchical federated diffusion model training framework. Methodologically: (1) it introduces task-similarity-driven client clustering and cross-cluster knowledge interaction; (2) integrates LoRA-based parameter-efficient fine-tuning, encrypted prompt encoding for secure transmission, and hierarchical aggregation to jointly optimize privacy preservation and communication efficiency; (3) enables local personalized adapter training alongside global multi-adapter co-evolution. Experiments demonstrate that the framework significantly accelerates convergence and enhances concurrent service capacity in edge environments, achieves generation quality comparable to centralized training, reduces communication overhead by 42%, maintains controllable privacy risks, and exhibits superior system scalability over existing federated AIGC approaches.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Agentic search
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Diffusion models (DMs) have emerged as powerful tools for high-quality content generation, yet their intensive computational requirements for inference pose challenges for resource-constrained edge devices. Cloud-based solutions aid in computation but often fall short in addressing privacy risks, personalization efficiency, and communication costs in multi-user edge-AIGC scenarios. To bridge this gap, we first analyze existing edge-AIGC applications in personalized content synthesis, revealing their limitations in efficiency and scalability. We then propose a novel cluster-aware hierarchical federated aggregation framework. Based on parameter-efficient local fine-tuning via Low-Rank Adaptation (LoRA), the framework first clusters clients based on the similarity of their uploaded task requirements, followed by an intra-cluster aggregation for enhanced personalization at the server-side. Subsequently, an inter-cluster knowledge interaction paradigm is implemented to enable hybrid-style content generation across diverse clusters.Building upon federated learning (FL) collaboration, our framework simultaneously trains personalized models for individual users at the devices and a shared global model enhanced with multiple LoRA adapters on the server,enabling efficient edge inference; meanwhile, all prompts for clustering and inference are encoded prior to transmission, thereby further mitigating the risk of plaintext leakage. Our evaluations demonstrate that the framework achieves accelerated convergence while maintaining practical viability for scalable multi-user personalized AIGC services under edge constraints.

Problem

• Efficient personalized AIGC for edge devices
• Privacy risks in multi-user edge-AIGC scenarios
• Scalability in edge-assisted content generation

Innovation

• Cluster-aware hierarchical federated aggregation framework
• Parameter-efficient local fine-tuning via LoRA
• Pre-encoded prompts for privacy protection