A Neuro-Symbolic System for Interpretable Multimodal Physiological Signals Integration in Human Fatigue Detection

2026-03-25

AI Summary

This study addresses the limited interpretability and weak individual diagnostic capability of existing models in human fatigue detection by proposing a neuro-symbolic architecture that integrates eye-tracking and functional near-infrared spectroscopy (fNIRS) signals. The model employs an attention-based encoder to learn four interpretable physiological concepts and combines them with differentiable logical reasoning based on Łukasiewicz operators for fatigue classification. A novel concept fidelity metric is introduced to enable individual-level model auditing and support participant-specific calibration. Evaluated via leave-one-subject-out cross-validation on 18 participants, the approach achieves an accuracy of 72.1% ± 12.3%. Concept fidelity shows a strong positive correlation with individual classification accuracy (r = 0.843, p < 0.0001), and participant-specific calibration further improves performance by 5.2 percentage points.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Neuro-Symbolic Learning
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

We propose a neuro-symbolic architecture that learns four interpretable physiological concepts, oculomotor dynamics, gaze stability, prefrontal hemodynamics, and multimodal, from eye-tracking and neural hemodynamics, functional near-infrared spectroscopy, (fNIRS) windows using attention-based encoders, and combines them with differentiable approximate reasoning rules using learned weights and soft thresholds, to address both rigid hand-crafted rules and the lack of subject-level alignment diagnostics. We apply this system to fatigue classification from multimodal physiological signals, a domain that requires models that are accurate and interpretable, with internal reasoning that can be inspected for safety-critical use. In leave-one-subject-out evaluation on 18 participants (560 samples), the method achieves 72.1% +/- 12.3% accuracy, comparable to tuned baselines while exposing concept activations and rule firing strengths. Ablations indicate gains from participant-specific calibration (+5.2 pp), a modest drop without the fNIRS concept (-1.2 pp), and slightly better performance with Lukasiewicz operators than product (+0.9 pp). We also introduce concept fidelity, an offline per-subject audit metric from held-out labels, which correlates strongly with per-subject accuracy (r=0.843, p < 0.0001).

Problem

• fatigue detection
• multimodal physiological signals
• interpretability
• neuro-symbolic system
• subject-level alignment

Innovation

• neuro-symbolic
• interpretable AI
• multimodal physiological signals
• differentiable reasoning
• concept fidelity