Context-Guided Semantic Alignment for Feature Fusion Networks

2026-06-11

AI Summary

This work addresses the semantic inconsistency arising from heterogeneous representations in multi-scale feature fusion, which limits object detection accuracy. To this end, the authors propose the FINE module, which leverages high-level contextual guidance to align low-level features through cross-level attention prior to fusion. An alignment-aware token sampling strategy is introduced to substantially reduce computational complexity. Furthermore, spatial-channel modulation combined with residual element-wise modulation is incorporated to enhance responses of semantically relevant pixels while preserving precise localization capabilities. The proposed method demonstrates consistent and significant performance gains across diverse detectors, achieving notable improvements in detection accuracy with negligible additional computational overhead.

Abstract

Feature fusion networks are fundamental components in modern object detectors, aggregating multi-scale features to detect objects of varying sizes. However, directly fusing features from different pyramid levels often introduces semantic inconsistency due to their heterogeneous representations. In this paper, we propose Feature Interaction NEtwork (FINE), a lightweight semantic alignment module that refines low-level features via high-level contextual guidance using cross-level attention prior to fusion. To bridge the structural gap and ensure computational efficiency, we introduce an Alignment-Aware Token Sampling that aligns corresponding spatial regions across scales, reducing the attention complexity by an order of magnitude. The resulting attention weights generate a spatial-channel modulation map that is upsampled and applied to the low-level features via residual element-wise modulation. This mechanism ensures that the network selectively enhances semantically relevant pixels while preserving the sub-pixel localization accuracy necessary for dense prediction tasks. FINE is generally applicable to various detectors and consistently improves detection accuracy without compromising efficiency.

Problem

• feature fusion
• semantic inconsistency
• multi-scale features
• object detection
• heterogeneous representations

Innovation

• semantic alignment
• cross-level attention
• feature fusion
• token sampling
• modulation map