Multi-Resolution Model Fusion for Accelerating the Convolutional Neural Network Training

2025-10-29

AI Summary

To address the prohibitively long training time of convolutional neural networks (CNNs) on high-dimensional large-array scientific data, this paper proposes a multi-resolution model fusion framework. First, lightweight submodels are rapidly pre-trained on progressively downsampled low-resolution representations. Subsequently, knowledge from these low-resolution models is transferred to the full-resolution model via a parameter fusion strategy. Finally, the fused model is fine-tuned at the original resolution. Crucially, we introduce a novel resolution-progressive fusion mechanism that preserves model accuracy while substantially accelerating training. Evaluated on two representative scientific computing benchmarks—CosmoFlow and Neuron Inverter—the method reduces end-to-end training time by 47% and 44%, respectively, without any accuracy degradation. This work establishes an effective trade-off between computational cost and model performance, offering a new paradigm for efficient deep learning on high-dimensional scientific datasets.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Neural networks are rapidly gaining popularity in scientific research, but training the models is often very time-consuming. Particularly when the training data samples are large high-dimensional arrays, efficient training methodologies that can reduce the computational costs are crucial. To reduce the training cost, we propose a Multi-Resolution Model Fusion (MRMF) method that combines models trained on reduced-resolution data and then refined with data in the original resolution. We demonstrate that these reduced-resolution models and datasets could be generated quickly. More importantly, the proposed approach reduces the training time by speeding up the model convergence in each fusion stage before switching to the final stage of finetuning with data in its original resolution. This strategy ensures the final model retains high-resolution insights while benefiting from the computational efficiency of lower-resolution training. Our experiment results demonstrate that the multi-resolution model fusion method can significantly reduce end-to-end training time while maintaining the same model accuracy. Evaluated using two real-world scientific applications, CosmoFlow and Neuron Inverter, the proposed method improves the training time by up to 47% and 44%, respectively, as compared to the original resolution training, while the model accuracy is not affected.

Problem

• Accelerating CNN training by reducing computational costs
• Fusing multi-resolution models to speed up convergence
• Maintaining accuracy while cutting training time significantly

Innovation

• Combines models trained on reduced-resolution data first
• Refines models using original high-resolution data later
• Speeds up convergence through multi-resolution fusion stages