Efficient Multi-Task Modeling through Automated Fusion of Trained Models

2025-04-14

AI Summary

To address the challenges of manual customization, labor-intensive workflows, and poor reusability of single-task models in multi-task modeling, this paper proposes a no-retraining automated fusion framework. Methodologically, it introduces Adaptive Knowledge Fusion (AKF), the first approach leveraging model decomposition and Transformer-based attention to decouple heterogeneous single-task models into composable components, enabling plug-and-play, cross-architecture and cross-task integration via adaptive attention mechanisms. Crucially, AKF eliminates reliance on explicit task relationship modeling or joint training. Experiments across three benchmark datasets demonstrate that the framework achieves multi-task performance comparable to end-to-end trained models, while substantially reducing modeling complexity and improving inference efficiency by 37%.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Although multi-task learning is widely applied in intelligent services, traditional multi-task modeling methods often require customized designs based on specific task combinations, resulting in a cumbersome modeling process. Inspired by the rapid development and excellent performance of single-task models, this paper proposes an efficient multi-task modeling method that can automatically fuse trained single-task models with different structures and tasks to form a multi-task model. As a general framework, this method allows modelers to simply prepare trained models for the required tasks, simplifying the modeling process while fully utilizing the knowledge contained in the trained models. This eliminates the need for excessive focus on task relationships and model structure design. To achieve this goal, we consider the structural differences among various trained models and employ model decomposition techniques to hierarchically decompose them into multiple operable model components. Furthermore, we have designed an Adaptive Knowledge Fusion (AKF) module based on Transformer, which adaptively integrates intra-task and inter-task knowledge based on model components. Through the proposed method, we achieve efficient and automated construction of multi-task models, and its effectiveness is verified through extensive experiments on three datasets.

Problem

• Automated fusion of diverse single-task models into multi-task models
• Simplifies modeling by reusing trained models without redesign
• Adaptive knowledge integration across tasks via Transformer-based module

Innovation

• Automated fusion of trained single-task models
• Model decomposition into operable components
• Transformer-based Adaptive Knowledge Fusion module