WQ-Fusion: Dynamic Gated Attention for Cross-Domain Audio Representation

2026-06-24

AI Summary

This work addresses the limited cross-acoustic-domain generalization of existing pretrained audio models by proposing WQ-Fusion, a dual-encoder framework that dynamically integrates heterogeneous features from Whisper (acoustic) and Qwen (semantic) through an adaptive feature modulation mechanism and element-wise gated attention. Unlike conventional static concatenation strategies, the proposed approach employs learnable gating to enable selective enhancement and adaptive fusion of cross-modal information. Evaluated on Track A of the Interspeech 2026 Audio Encoder Capability Challenge, the method achieves a state-of-the-art composite score of 0.836, significantly outperforming the strongest single-encoder baseline and demonstrating its effectiveness and superiority.

Abstract

While pre-trained models excel in specialized tasks, learning universal representations across diverse acoustic domains remains challenging. To address this, we propose WQ-Fusion, a robust dual-encoder framework for cross-domain audio representation learning. Overcoming the limitations of static concatenation, WQ-Fusion integrates whisper and qwen via an Adaptive Feature Modulation module and a novel element-wise gated attention mechanism. This design enables dynamic feature selection, allowing the model to selectively emphasize relevant acoustic and semantic dimensions. Extensive experiments on the Interspeech 2026 Audio Encoder Capability Challenge (Track A) benchmark demonstrate that by effectively routing heterogeneous information, WQ-Fusion achieves a superior overall score of 0.836, significantly outperforming the strongest single-encoder baseline.

Problem

• cross-domain audio representation
• universal representations
• acoustic domains
• audio encoder

Innovation

• Dynamic Gated Attention
• Cross-Domain Audio Representation
• Adaptive Feature Modulation
• Dual-Encoder Fusion
• Universal Audio Embedding