FAConformer: Frequency-Aware Convolutional Transformer for Auditory Attention Decoding

2026-06-12

AI Summary

This work addresses the challenge that existing auditory attention decoding (AAD) models struggle to effectively capture both band-specific patterns and cross-frequency interactions in electroencephalography (EEG) signals. To overcome this limitation, the authors propose FAConformer, a novel framework that first decomposes EEG into multiple frequency bands and employs dedicated CNN-Transformer encoders for band-specific modeling. A frequency-aware attention module is then introduced to adaptively fuse cross-band features, complemented by a band-level auxiliary supervision strategy to enhance training efficacy in weakly contributing branches. FAConformer represents the first deep frequency-domain modeling approach in AAD, consistently outperforming twelve baseline models across two public datasets and three decision window lengths, surpassing the current state-of-the-art by 4.9% and demonstrating strong effectiveness, robustness, and interpretability.

Abstract

Auditory attention decoding (AAD) aims to infer the attended speaker from neural responses in multi-speaker acoustic environments and is a key problem for neuro-steered hearing systems. Although recent studies have achieved encouraging progress, existing AAD models still do not fully exploit frequency domain electroencephalography (EEG) information. In particular, most approaches introduce multi-band information through handcrafted feature extraction or direct cross-band feature concatenation, which mainly exploit frequency information at a shallow level and may overlook band-specific patterns and cross-band interactions. To address these limitations, this paper proposes FAConformer, a frequency-aware CNN-Transformer framework for AAD that explicitly integrates band-specific encoding and adaptive cross-band interaction. Specifically, FAConformer first decomposes EEG signals into multiple frequency bands and assigns each band to an independent CNN-Transformer encoder for band-specific modeling. The resulting band-wise features are then adaptively fused by a carefully designed frequency-aware attention (FAA) module that models cross-band dependencies by treating band-wise features as tokens. Further, band-wise auxiliary supervision (BAS) is introduced to prevent weakly contributing branches from being under-optimized during joint training. In this way, FAConformer performs frequency-aware modeling that more effectively exploits frequency domain information. Extensive experiments on two public AAD datasets with three decision-window lengths demonstrated that FAConformer consistently outperformed 12 competitive baselines, surpassing the current state-of-the-art model by 4.9%. Further analyses of band importance, ablation, and parameter sensitivity verify the effectiveness, robustness, and interpretability of the proposed framework. Code is available at https://github.com/wzwvv/FAConformer.

Problem

• Auditory Attention Decoding
• Frequency Domain
• EEG
• Multi-band Information
• Cross-band Interaction

Innovation

• Frequency-Aware Attention
• Band-Specific Encoding
• Cross-Band Interaction
• CNN-Transformer
• Auditory Attention Decoding