Uncertainty Quantification for Motor Imagery BCI -- Machine Learning vs. Deep Learning

2025-07-10

AI Summary

Existing motion imagery brain–computer interfaces (MI-BCIs) lack reliable uncertainty quantification for classifier predictions, limiting trustworthiness and safety. Method: We systematically benchmark conventional approaches (CSP-LDA, MDRM) against deep learning methods (Deep Ensembles, direct uncertainty quantification, CNN) on confidence estimation. We propose MDRM-T, a temperature-based calibration strategy that mitigates MDRM’s underconfidence. Contribution/Results: CSP-LDA and calibrated MDRM-T achieve optimal uncertainty calibration, while Deep Ensembles and CNN yield highest classification accuracy. All models effectively discriminate easy from hard samples, enabling confidence-based rejection. Integrating rejection improves both overall accuracy and reliability. This work presents the first comprehensive evaluation and enhancement of uncertainty modeling in traditional MI-BCI classifiers, providing empirical foundations and practical solutions for trustworthy BCI design.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Brain-computer interfaces (BCIs) turn brain signals into functionally useful output, but they are not always accurate. A good Machine Learning classifier should be able to indicate how confident it is about a given classification, by giving a probability for its classification. Standard classifiers for Motor Imagery BCIs do give such probabilities, but research on uncertainty quantification has been limited to Deep Learning. We compare the uncertainty quantification ability of established BCI classifiers using Common Spatial Patterns (CSP-LDA) and Riemannian Geometry (MDRM) to specialized methods in Deep Learning (Deep Ensembles and Direct Uncertainty Quantification) as well as standard Convolutional Neural Networks (CNNs). We found that the overconfidence typically seen in Deep Learning is not a problem in CSP-LDA and MDRM. We found that MDRM is underconfident, which we solved by adding Temperature Scaling (MDRM-T). CSP-LDA and MDRM-T give the best uncertainty estimates, but Deep Ensembles and standard CNNs give the best classifications. We show that all models are able to separate between easy and difficult estimates, so that we can increase the accuracy of a Motor Imagery BCI by rejecting samples that are ambiguous.

Problem

• Compare uncertainty quantification in Motor Imagery BCI classifiers
• Evaluate Machine Learning vs Deep Learning confidence estimation
• Improve BCI accuracy by rejecting ambiguous samples

Innovation

• Compares CSP-LDA and MDRM with Deep Learning methods
• Introduces Temperature Scaling for MDRM (MDRM-T)
• Rejects ambiguous samples to improve BCI accuracy