In-Vivo Training for Deep Brain Stimulation

2025-10-03

AI Summary

Existing RL-based deep brain stimulation (DBS) approaches rely on in vivo unmeasurable surrogate biomarkers, hindering clinical translation. This work proposes the first online adaptive DBS framework grounded exclusively in clinically accessible neural signals—specifically, β-band oscillatory power—recorded from cortico-basal ganglia local field potentials. Leveraging the TD3 reinforcement learning algorithm and integrated with a biologically constrained basal ganglia computational model, the framework dynamically optimizes stimulation parameters in real time within a closed-loop configuration. It enables patient-specific online training without requiring prior physiological modeling or offline calibration. Multiscale simulations demonstrate that the method achieves significantly superior suppression of Parkinson’s disease–associated pathological biomarkers compared to both conventional fixed-parameter and reactive DBS strategies. This study is the first to empirically validate the feasibility and superiority of efficient, robust, and personalized closed-loop neuromodulation using only intracranially measurable signals.

Technology Category

• Machine Learning: Online Learning & Bandits
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Deep Brain Stimulation (DBS) is a highly effective treatment for Parkinson's Disease (PD). Recent research uses reinforcement learning (RL) for DBS, with RL agents modulating the stimulation frequency and amplitude. But, these models rely on biomarkers that are not measurable in patients and are only present in brain-on-chip (BoC) simulations. In this work, we present an RL-based DBS approach that adapts these stimulation parameters according to brain activity measurable in vivo. Using a TD3 based RL agent trained on a model of the basal ganglia region of the brain, we see a greater suppression of biomarkers correlated with PD severity compared to modern clinical DBS implementations. Our agent outperforms the standard clinical approaches in suppressing PD biomarkers while relying on information that can be measured in a real world environment, thereby opening up the possibility of training personalized RL agents specific to individual patient needs.

Problem

• Developing RL-based DBS using measurable in-vivo brain activity
• Overcoming reliance on non-measurable biomarkers from simulations
• Creating personalized DBS treatments adaptable to individual patients

Innovation

• Uses TD3 reinforcement learning for DBS
• Adapts stimulation to in-vivo brain activity
• Trains personalized agents for individual patients