Adaptive Anisotropic Attention for Axis-Structured Signals

2026-09-08

AI Summary

为了解决EEG信号中均匀先验导致的无关交互问题，提出自适应各向异性注意力(AAA)，通过时间路径和空间路径分别处理，并用门控机制结合两者输出。

Abstract

Dense self-attention treats all token pairs as equally plausible before learning, an interaction-isotropic prior that can be mismatched to structured signals. For structured, low signal-to-noise ratio (SNR) signals such as EEG, dependencies are organized along the electrode and time axes, and this uniform prior exposes each token to many irrelevant interactions. We introduce Adaptive Anisotropic Attention (AAA), which splits attention into two paths: a temporal path, where each token attends to the tokens of its own electrode across time, and a spatial path, where it attends to the tokens of the other electrodes at the same time step. A small gate predicts, for every token, a convex combination of the two path outputs: two non-negative weights that sum to one. On six EEG downstream tasks, the resulting model, AXON (AXis-factorized Operator Network), improves mean balanced accuracy over a dense baseline under both linear probing and full fine-tuning. We show that both paths (temporal and spatial) are necessary and that the weighted sum beats a hard choice of one path; most of the benefit comes from the gate learning a different temporal/spatial balance at each layer of the network. Controlled audio spectrogram experiments show that axis factorization transfers beyond EEG. These results suggest that aligning attention with the natural axes of structured signals provides a useful inductive bias.

Problem

• Adaptive Anisotropic Attention
• structured signals
• signal-to-noise ratio (SNR)
• EEG
• attention mechanism

Innovation

• Adaptive Anisotropic Attention
• temporal path
• spatial path
• gate mechanism
• axis factorization