CUT: Pruning Pre-Trained Multi-Task Models into Compact Models for Edge Devices

2025-04-14

AI Summary

To address the challenges of deploying large, task-redundant pre-trained multi-task models on resource-constrained edge devices, this paper proposes a task-aware structured pruning method. Unlike prior approaches, it jointly models task selection, parameter importance estimation, and cross-task parameter fusion—enabling on-demand extraction and recombination of task-relevant parameters to yield lightweight, customized multi-task submodels while preserving essential knowledge. Key technical contributions include: (1) a gradient-activation joint importance scoring scheme for pruning; (2) a shared-specific parameter fusion mechanism tailored for multi-task learning; and (3) task decoupling and dynamic recombination techniques. Extensive experiments on three image benchmark datasets demonstrate that the pruned models achieve up to 68% parameter reduction, 2.3× inference speedup, and less than 1.2% accuracy degradation—substantially outperforming existing baselines.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multi-task learning has garnered widespread attention in the industry due to its efficient data utilization and strong generalization capabilities, making it particularly suitable for providing high-quality intelligent services to users. Edge devices, as the primary platforms directly serving users, play a crucial role in delivering multi-task services. However, current multi-task models are often large, and user task demands are increasingly diverse. Deploying such models directly on edge devices not only increases the burden on these devices but also leads to task redundancy. To address this issue, this paper innovatively proposes a pre-trained multi-task model pruning method specifically designed for edge computing. The goal is to utilize existing pre-trained multi-task models to construct a compact multi-task model that meets the needs of edge devices. The specific implementation steps are as follows: First, decompose the tasks within the pre-trained multi-task model and select tasks based on actual user needs. Next, while retaining the knowledge of the original pre-trained model, evaluate parameter importance and use a parameter fusion method to effectively integrate shared parameters among tasks. Finally, obtain a compact multi-task model suitable for edge devices. To validate the effectiveness of the proposed method, we conducted experiments on three public image datasets. The experimental results fully demonstrate the superiority and efficiency of this method, providing a new solution for multi-task learning on edge devices.

Problem

• Prune large multi-task models for edge devices
• Reduce task redundancy in edge computing
• Create compact models from pre-trained multi-task models

Innovation

• Prunes pre-trained multi-task models for edge devices
• Selects tasks based on actual user needs
• Uses parameter fusion to integrate shared parameters