A Multi-Camera Optical Tag Neuronavigation and AR Augmentation Framework for Non-Invasive Brain Stimulation

2026-01-28

AI Summary

This study addresses the limitations of conventional transcranial magnetic stimulation (TMS) neuronavigation, which relies on costly and error-prone dedicated tracking hardware, hindering precise and convenient coil placement. To overcome this, the authors propose a novel navigation system integrating low-cost multi-view vision, self-coordinating optical markers, and augmented reality (AR). The system leverages multiple cameras to estimate in real time the 3D poses of both the patient’s head and the TMS coil, and fuses this information with a dynamic digital twin brain model to deliver an intuitive, immersive navigation experience via AR headsets or mobile devices. Eliminating the need for specialized tracking equipment, the approach significantly outperforms existing methods in spatial accuracy and usability. Clinical evaluation by ten medically trained participants confirmed its high practical utility and deployment potential.

Technology Category

• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Accurate neuronavigation is essential for generating the intended effect with transcranial magnetic stimulation (TMS). Precise coil placement also directly influences stimulation efficacy. Traditional neuronavigation systems often rely on costly and still hard to use and error-prone tracking systems. To solve these limitations, we present a computer-vision-based neuronavigation system for real-time tracking of patient and TMS instrumentation. The system can feed the necessary data for a digital twin to track TMS stimulation targets. We integrate a self-coordinating optical tracking system with multiple consumer-grade cameras and visible tags with a dynamic 3D brain model in Unity. This model updates in real time to represent the current stimulation coil position and the estimated stimulation point to intuitively visualize neural targets for clinicians. We incorporate an augmented reality (AR) module to bridge the gap between the visualization of the digital twin and the real world and project the brain model in real-time onto the head of a patient. AR headsets or mobile AR devices allow clinicians to interactively view and adjust the placement of the stimulation transducer intuitively instead of guidance through abstract numbers and 6D cross hairs on an external screen. The proposed technique provides improved spatial precision as well as accuracy. A case study with ten participants with a medical background also demonstrates that the system has high usability.

Problem

• neuronavigation
• transcranial magnetic stimulation
• optical tracking
• augmented reality
• coil placement

Innovation

• computer vision
• augmented reality
• neuronavigation
• optical tracking
• digital twin