Skewness-Guided Pruning of Multimodal Swin Transformers for Federated Skin Lesion Classification on Edge Devices

2025-12-09

AI Summary

To address the dual challenges of high computational overhead and stringent privacy constraints in deploying medical imaging models on edge devices, this paper proposes a federated learning–model compression co-design framework for skin lesion classification. Our method innovatively incorporates output distribution skewness into multimodal Swin Transformer pruning, enabling the first structured, statistics-driven pruning of both attention mechanisms and feed-forward networks, seamlessly integrated into a horizontal federated learning pipeline. Crucially, the compression is lossless—preserving model accuracy while significantly reducing resource demands. Experiments demonstrate a 36% reduction in model size, substantial inference acceleration, and robust classification performance under realistic federated settings. This work delivers a practical, privacy-preserving, and computationally efficient solution for edge-based medical image analysis in resource-constrained, privacy-sensitive environments.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

In recent years, high-performance computer vision models have achieved remarkable success in medical imaging, with some skin lesion classification systems even surpassing dermatology specialists in diagnostic accuracy. However, such models are computationally intensive and large in size, making them unsuitable for deployment on edge devices. In addition, strict privacy constraints hinder centralized data management, motivating the adoption of Federated Learning (FL). To address these challenges, this study proposes a skewness-guided pruning method that selectively prunes the Multi-Head Self-Attention and Multi-Layer Perceptron layers of a multimodal Swin Transformer based on the statistical skewness of their output distributions. The proposed method was validated in a horizontal FL environment and shown to maintain performance while substantially reducing model complexity. Experiments on the compact Swin Transformer demonstrate approximately 36% model size reduction with no loss in accuracy. These findings highlight the feasibility of achieving efficient model compression and privacy-preserving distributed learning for multimodal medical AI on edge devices.

Problem

• Reduces model size for edge deployment
• Maintains accuracy in federated learning
• Compresses multimodal medical AI efficiently

Innovation

• Skewness-guided pruning reduces model complexity
• Selective pruning targets attention and MLP layers
• Maintains accuracy while shrinking model size