Enabling Flexible Multi-LLM Integration for Scalable Knowledge Aggregation

2025-05-28

AI Summary

Large language models (LLMs) face significant challenges in multi-model ensembling, including excessive memory overhead, inflexible weight merging, severe knowledge interference, and degraded task performance. Method: This paper proposes an adaptive multi-LLM ensemble framework featuring a novel feedback-regulated selection network and a dynamic weighted fusion mechanism, enabling task-aware dynamic selection of heterogeneous models and knowledge aggregation; it further introduces a feedback-driven loss function and a collaborative distillation strategy across multiple LLMs to mitigate knowledge conflicts. Contribution/Results: Experiments demonstrate that the method reduces knowledge interference by 50%, substantially improving ensemble stability, scalability, and multi-task robustness. Crucially, it achieves these gains without full-parameter fine-tuning, effectively circumventing the memory and adaptability bottlenecks inherent in conventional ensembling approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language models (LLMs) have shown remarkable promise but remain challenging to continually improve through traditional finetuning, particularly when integrating capabilities from other specialized LLMs. Popular methods like ensemble and weight merging require substantial memory and struggle to adapt to changing data environments. Recent efforts have transferred knowledge from multiple LLMs into a single target model; however, they suffer from interference and degraded performance among tasks, largely due to limited flexibility in candidate selection and training pipelines. To address these issues, we propose a framework that adaptively selects and aggregates knowledge from diverse LLMs to build a single, stronger model, avoiding the high memory overhead of ensemble and inflexible weight merging. Specifically, we design an adaptive selection network that identifies the most relevant source LLMs based on their scores, thereby reducing knowledge interference. We further propose a dynamic weighted fusion strategy that accounts for the inherent strengths of candidate LLMs, along with a feedback-driven loss function that prevents the selector from converging on a single subset of sources. Experimental results demonstrate that our method can enable a more stable and scalable knowledge aggregation process while reducing knowledge interference by up to 50% compared to existing approaches. Code is avaliable at https://github.com/ZLKong/LLM_Integration

Problem

• Enabling flexible integration of multiple LLMs for scalable knowledge aggregation
• Reducing memory overhead and interference in multi-LLM knowledge transfer
• Improving adaptability and performance in dynamic data environments

Innovation

• Adaptive selection network for relevant LLMs
• Dynamic weighted fusion for candidate strengths
• Feedback-driven loss prevents subset convergence