In-Model Merging for Enhancing the Robustness of Medical Imaging Classification Models

2025-02-27

AI Summary

To address the insufficient robustness of medical image classification models and the inference overhead introduced by conventional model ensembling, this paper proposes InMerge—the first method to perform online, dynamic fusion of similar convolutional kernels *within* a single CNN during training. Its core contributions are: (1) layer-wise adaptive kernel merging guided by similarity metrics; (2) a gradient-coordinated parameter update mechanism ensuring post-merging parameters remain differentiable and optimizable; and (3) a multi-architecture adaptation strategy enhancing generalizability across diverse CNN backbones. Experiments on four benchmark medical imaging datasets demonstrate that InMerge significantly improves both model robustness and generalization—without increasing inference latency or computational cost. Furthermore, this work identifies, for the first time, key architectural properties governing kernel fusion efficacy, establishing a novel paradigm for lightweight, robust AI in medical imaging.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Machine Learning: Kernel Methods
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Model merging is an effective strategy to merge multiple models for enhancing model performances, and more efficient than ensemble learning as it will not introduce extra computation into inference. However, limited research explores if the merging process can occur within one model and enhance the model's robustness, which is particularly critical in the medical image domain. In the paper, we are the first to propose in-model merging (InMerge), a novel approach that enhances the model's robustness by selectively merging similar convolutional kernels in the deep layers of a single convolutional neural network (CNN) during the training process for classification. We also analytically reveal important characteristics that affect how in-model merging should be performed, serving as an insightful reference for the community. We demonstrate the feasibility and effectiveness of this technique for different CNN architectures on 4 prevalent datasets. The proposed InMerge-trained model surpasses the typically-trained model by a substantial margin. The code will be made public.

Problem

• Enhancing robustness of medical imaging classification models
• Selectively merging convolutional kernels within a single CNN
• Improving model performance without extra inference computation

Innovation

• In-model merging enhances CNN robustness.
• Selective kernel merging in deep layers.
• Improves medical imaging classification performance.