Redundancy-optimized Multi-head Attention Networks for Multi-View Multi-Label Feature Selection

2025-11-16

AI Summary

Existing attention mechanisms for multi-view multi-label learning neglect inter-view complementarity, inadequately model feature–label correlations, and lack explicit redundancy suppression. To address these issues, this paper proposes an Optimization-Driven Redundancy-aware Multi-head Cross-View Attention Network (OR-MCAN). Methodologically, OR-MCAN introduces a cross-view attention mechanism to capture complementary information across views, employs a query-key correlation-driven module for joint feature–label representation learning, and integrates both static and dynamic redundancy regularization terms to explicitly suppress redundant features while enhancing subset compactness and diversity. Extensive experiments on six real-world benchmark datasets demonstrate that OR-MCAN consistently outperforms six state-of-the-art baselines across multiple metrics—achieving superior performance in feature selection quality, multi-label classification accuracy, and model stability—thereby establishing new state-of-the-art results.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Multi-view multi-label data offers richer perspectives for artificial intelligence, but simultaneously presents significant challenges for feature selection due to the inherent complexity of interrelations among features, views and labels. Attention mechanisms provide an effective way for analyzing these intricate relationships. They can compute importance weights for information by aggregating correlations between Query and Key matrices to focus on pertinent values. However, existing attention-based feature selection methods predominantly focus on intra-view relationships, neglecting the complementarity of inter-view features and the critical feature-label correlations. Moreover, they often fail to account for feature redundancy, potentially leading to suboptimal feature subsets. To overcome these limitations, we propose a novel method based on Redundancy-optimized Multi-head Attention Networks for Multi-view Multi-label Feature Selection (RMAN-MMFS). Specifically, we employ each individual attention head to model intra-view feature relationships and use the cross-attention mechanisms between different heads to capture inter-view feature complementarity. Furthermore, we design static and dynamic feature redundancy terms: the static term mitigates redundancy within each view, while the dynamic term explicitly models redundancy between unselected and selected features across the entire selection process, thereby promoting feature compactness. Comprehensive evaluations on six real-world datasets, compared against six multi-view multi-label feature selection methods, demonstrate the superior performance of the proposed method.

Problem

• Optimizing feature selection for multi-view multi-label data
• Addressing inter-view complementarity and feature-label correlations
• Reducing feature redundancy through static and dynamic mechanisms

Innovation

• Multi-head attention networks model intra-view relationships
• Cross-attention mechanisms capture inter-view feature complementarity
• Static and dynamic terms optimize feature redundancy reduction