MHAF-YOLO: Multi-Branch Heterogeneous Auxiliary Fusion YOLO for accurate object detection

2025-02-07

AI Summary

To address the limitation of PAFPN in YOLO architectures—namely, its difficulty in simultaneously preserving high-level semantic information and low-level spatial details, leading to degraded detection performance under scale variation (especially for small objects)—this paper proposes MHAF-YOLO. Its core is a multi-branch heterogeneous auxiliary fusion neck, termed MAFPN, comprising a shallow spatial fidelity module (SAF) and a deep gradient enhancement module (AAF). Additionally, we introduce two novel components: global heterogeneous flexible kernel selection (GHFKS) and reparameterized hierarchical multi-scale convolution (RepHMS), which jointly expand receptive fields both vertically and horizontally. This design significantly enhances multi-scale feature representation capability. Extensive experiments demonstrate that MHAF-YOLO surpasses mainstream YOLO variants on standard benchmarks such as COCO, particularly improving small-object detection accuracy. The source code is publicly available.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Large Vision Models
• Multiagent Systems: Multiagent Planning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Due to the effective multi-scale feature fusion capabilities of the Path Aggregation FPN (PAFPN), it has become a widely adopted component in YOLO-based detectors. However, PAFPN struggles to integrate high-level semantic cues with low-level spatial details, limiting its performance in real-world applications, especially with significant scale variations. In this paper, we propose MHAF-YOLO, a novel detection framework featuring a versatile neck design called the Multi-Branch Auxiliary FPN (MAFPN), which consists of two key modules: the Superficial Assisted Fusion (SAF) and Advanced Assisted Fusion (AAF). The SAF bridges the backbone and the neck by fusing shallow features, effectively transferring crucial low-level spatial information with high fidelity. Meanwhile, the AAF integrates multi-scale feature information at deeper neck layers, delivering richer gradient information to the output layer and further enhancing the model learning capacity. To complement MAFPN, we introduce the Global Heterogeneous Flexible Kernel Selection (GHFKS) mechanism and the Reparameterized Heterogeneous Multi-Scale (RepHMS) module to enhance feature fusion. RepHMS is globally integrated into the network, utilizing GHFKS to select larger convolutional kernels for various feature layers, expanding the vertical receptive field and capturing contextual information across spatial hierarchies. Locally, it optimizes convolution by processing both large and small kernels within the same layer, broadening the lateral receptive field and preserving crucial details for detecting smaller targets. The source code of this work is available at: https://github.com/yang0201/MHAF-YOLO.

Problem

• Enhance multi-scale feature fusion
• Improve object detection accuracy
• Address significant scale variations

Innovation

• Multi-Branch Auxiliary FPN
• Global Heterogeneous Flexible Kernel
• Reparameterized Heterogeneous Multi-Scale