Neural Signals Generate Clinical Notes in the Wild

2026-01-29

AI Summary

This study addresses the lack of automated tools for clinical electroencephalogram (EEG) report generation, which currently relies heavily on manual interpretation. To this end, the authors construct the first large-scale paired dataset of clinical EEG recordings and corresponding narrative reports and propose CELM, an end-to-end multimodal foundation model that enables scalable joint modeling of EEG signals and language. CELM integrates a pretrained EEG encoder with a large language model, facilitating zero-shot, multi-scale report generation without requiring patient history. Experimental results demonstrate that CELM achieves a 70%–95% relative improvement over existing baselines on standard metrics, with zero-shot ROUGE-1 and METEOR scores reaching 0.43–0.52, significantly outperforming current approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Generating clinical reports that summarize abnormal patterns, diagnostic findings, and clinical interpretations from long-term EEG recordings remains labor-intensive. We curate a large-scale clinical EEG dataset with $9{,}922$ reports paired with approximately $11{,}000$ hours of EEG recordings from $9{,}048$ patients. We therefore develop CELM, the first clinical EEG-to-Language foundation model capable of summarizing long-duration, variable-length EEG recordings and performing end-to-end clinical report generation at multiple scales, including recording description, background activity, epileptiform abnormalities, events/seizures, and impressions. Experimental results show that, with patient history supervision, our method achieves $70\%$--$95\%$ average relative improvements in standard generation metrics (e.g., ROUGE-1 and METEOR) from $0.2$--$0.3$ to $0.4$--$0.6$. In the zero-shot setting without patient history, CELM attains generation scores in the range of $0.43$--$0.52$, compared to baselines of $0.17$--$0.26$. CELM integrates pretrained EEG foundation models with language models to enable scalable multimodal learning. We release our model and benchmark construction pipeline at [URL].

Problem

• clinical report generation
• EEG interpretation
• neural signals
• long-term EEG
• automated summarization

Innovation

• EEG-to-language model
• clinical report generation
• multimodal foundation model
• long-duration EEG summarization
• zero-shot clinical interpretation