Multimodal Bayesian Network for Robust Assessment of Casualties in Autonomous Triage

2025-12-21

AI Summary

In mass-casualty incidents, existing automated triage systems suffer from delayed assessment and misclassification due to reliance on labeled training data, poor robustness, and limited interpretability. To address these limitations, we propose the first purely knowledge-driven, multimodal Bayesian triage framework. It fuses perception outputs from multiple vision models detecting critical signs (e.g., massive hemorrhage, respiratory distress), encodes clinical expert rules into a static, data-free Bayesian network, and integrates an uncertainty-aware inference engine enabling verifiable, interpretable decisions under noisy or incomplete observations. Crucially, the method eliminates dependence on annotated datasets. Evaluated on the DARPA Triage Challenge, it improves physiological assessment accuracy from 15%/19% to 42%/46%, overall triage accuracy from 14% to 53%, and diagnostic coverage from 31% to 95%, securing fourth place in the competition’s first round.

Technology Category

• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Mass Casualty Incidents can overwhelm emergency medical systems and resulting delays or errors in the assessment of casualties can lead to preventable deaths. We present a decision support framework that fuses outputs from multiple computer vision models, estimating signs of severe hemorrhage, respiratory distress, physical alertness, or visible trauma, into a Bayesian network constructed entirely from expert-defined rules. Unlike traditional data-driven models, our approach does not require training data, supports inference with incomplete information, and is robust to noisy or uncertain observations. We report performance for two missions involving 11 and 9 casualties, respectively, where our Bayesian network model substantially outperformed vision-only baselines during evaluation of our system in the DARPA Triage Challenge (DTC) field scenarios. The accuracy of physiological assessment improved from 15% to 42% in the first scenario and from 19% to 46% in the second, representing nearly threefold increase in performance. More importantly, overall triage accuracy increased from 14% to 53% in all patients, while the diagnostic coverage of the system expanded from 31% to 95% of the cases requiring assessment. These results demonstrate that expert-knowledge-guided probabilistic reasoning can significantly enhance automated triage systems, offering a promising approach to supporting emergency responders in MCIs. This approach enabled Team Chiron to achieve 4th place out of 11 teams during the 1st physical round of the DTC.

Problem

• Develops a Bayesian network for automated casualty assessment in mass emergencies.
• Fuses multiple vision models to detect hemorrhage, distress, and trauma signs.
• Improves triage accuracy without training data using expert-defined rules.

Innovation

• Bayesian network fuses multiple computer vision outputs
• Expert-defined rules replace traditional data-driven training
• Robust inference with incomplete and noisy observations