Dual-Attention Convolution Experts for Sparse Tensor Completion

2026-06-19

AI Summary

This work addresses the challenges of modeling complex cross-modal interactions and extreme sparsity in high-dimensional sparse tensor completion by proposing the DCGC method. DCGC integrates a multi-channel convolutional network to generate nonlinear alignment patterns, employs a gated dual-attention mechanism to dynamically emphasize critical channels and features, and introduces group-level contrastive learning to strengthen self-supervised signals and alleviate data sparsity. By embedding these components into a neural tensor factorization framework, DCGC achieves substantially higher completion accuracy than state-of-the-art methods across five benchmark datasets in traffic and recommendation domains, while preserving strong model generalization capabilities.

Abstract

Tensor factorization (TF) has been widely adopted for high-dimensional sparse data completion tasks. Despite significant progress, neural TF methods often struggle to capture complex cross-mode interactions and remain vulnerable to (extreme) data sparsity. To address these challenges, we propose a novel neural tensor factorization approach, termed Dual-Attention Convolution Expert Networks with Group-Level Contrastive Learning (DCGC). For the first problem, DCGC generates diverse non-linear alignment patterns of latent factors via a multi-channel convolution network, and leverages the gated dual-attention mechanism to drive the model to focus on more important output channels (i.e., convolution experts) and the aligned features. Furthermore, DCGC introduces a group-level contrastive learning strategy that aggregates positive samples with identical feedback levels while separating negative samples across different levels. This strategy injects high-quality self-supervised signals to mitigate data sparsity. Extensive experiments conducted on five datasets demonstrate that our DCGC outperforms the state-of-the-art methods in sparse tensor completion for traffic and recommendation applications. Code to reproduce the experimental results in the paper is available at https://github.com/ku1z/DCGC.

Problem

• tensor factorization
• sparse tensor completion
• cross-mode interactions
• data sparsity
• neural TF

Innovation

• Dual-Attention Mechanism
• Convolution Experts
• Group-Level Contrastive Learning
• Sparse Tensor Completion
• Neural Tensor Factorization