LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis

2025-10-25

AI Summary

To address limited model generalizability across EEG devices caused by heterogeneous electrode layouts (i.e., topological inconsistency), this paper proposes LUNA, a topology-agnostic foundation model. Methodologically, LUNA introduces, for the first time, a latent-space modeling mechanism that requires no prior knowledge of electrode topology: it compresses multi-channel EEG into fixed-dimensional representations via learnable queries and cross-attention, achieving linear computational complexity and strong cross-device transferability. It employs masked segment reconstruction for self-supervised pretraining, integrating a cross-attention encoder with a block-wise temporal self-attention decoder to efficiently capture temporal dynamics. Evaluated on four downstream tasks—including anomaly detection and artifact removal—LUNA achieves state-of-the-art performance, attaining an AUROC of 0.921 on TUAR. It reduces computational cost by 300× and GPU memory usage to 1/10 of prior methods, while remaining fully compatible with arbitrary electrode configurations.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Electroencephalography (EEG) offers a non-invasive lens into human brain activity, but building large-scale models is hampered by topological heterogeneity: each public EEG data defines its own electrode layout, limiting generalization. We introduce LUNA (Latent Unified Network Architecture), a self-supervised foundation model that reconciles disparate electrode geometries while scaling linearly -- not quadratically -- with channel count. LUNA compresses multi-channel EEG into a fixed-size, topology-agnostic latent space via learned queries and cross-attention. Downstream transformer blocks then operate exclusively on this latent representation using patch-wise temporal self-attention, decoupling computation from electrode count. Pre-trained on TUEG and Siena (over 21,000 hours of raw EEG across diverse montages) using a masked-patch reconstruction objective, LUNA transfers effectively to four downstream tasks: abnormality detection, artifact rejection, slowing classification, and emotion recognition. It demonstrates highly competitive performance across several benchmarks, achieving state-of-the-art results on TUAR and TUSL, e.g., 0.921 AUROC on TUAR, while reducing FLOPs by 300x and trimming GPU memory use by up to 10x. Critically, these gains are consistent across all evaluated electrode configurations. Code is available at https://github.com/pulp-bio/BioFoundation

Problem

• Handling topological heterogeneity in EEG electrode layouts
• Scaling computational efficiency with increasing channel counts
• Enabling cross-dataset generalization for EEG analysis tasks

Innovation

• Compresses EEG into topology-agnostic latent space
• Uses cross-attention to handle varying electrode layouts
• Decouples computation from electrode count for efficiency