Exploring Synergistic Ensemble Learning: Uniting CNNs, MLP-Mixers, and Vision Transformers to Enhance Image Classification

2025-04-12

AI Summary

Single-architectural vision models face inherent performance bottlenecks in image classification. Method: This paper proposes a cross-paradigm ensemble framework that preserves architectural integrity by integrating three heterogeneous architectures—CNN (ResNet), MLP-Mixer, and Vision Transformer—via weighted averaging and majority voting, without architectural modification or feature-space alignment. Crucially, the framework implicitly isolates their respective feature spaces while enabling synergistic gains. Contribution/Results: It introduces the first complementary analysis paradigm grounded in architectural orthogonality. Evaluated end-to-end on ImageNet, the ensemble surpasses prior single-model SOTA in top-1 accuracy while reducing overall inference latency—establishing a new benchmark for efficient, high-accuracy image classification.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Ensemble Methods
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In recent years, Convolutional Neural Networks (CNNs), MLP-mixers, and Vision Transformers have risen to prominence as leading neural architectures in image classification. Prior research has underscored the distinct advantages of each architecture, and there is growing evidence that combining modules from different architectures can boost performance. In this study, we build upon and improve previous work exploring the complementarity between different architectures. Instead of heuristically merging modules from various architectures through trial and error, we preserve the integrity of each architecture and combine them using ensemble techniques. By maintaining the distinctiveness of each architecture, we aim to explore their inherent complementarity more deeply and with implicit isolation. This approach provides a more systematic understanding of their individual strengths. In addition to uncovering insights into architectural complementarity, we showcase the effectiveness of even basic ensemble methods that combine models from diverse architectures. These methods outperform ensembles comprised of similar architectures. Our straightforward ensemble framework serves as a foundational strategy for blending complementary architectures, offering a solid starting point for further investigations into the unique strengths and synergies among different architectures and their ensembles in image classification. A direct outcome of this work is the creation of an ensemble of classification networks that surpasses the accuracy of the previous state-of-the-art single classification network on ImageNet, setting a new benchmark, all while requiring less overall latency.

Problem

• Enhancing image classification by combining CNNs, MLP-Mixers, and Vision Transformers
• Exploring complementarity between architectures via ensemble learning techniques
• Improving accuracy and latency in ImageNet classification with ensemble methods

Innovation

• Combines CNNs, MLP-Mixers, Vision Transformers via ensemble
• Preserves architecture integrity while exploring complementarity
• Basic ensemble methods outperform similar architecture ensembles