Personalizing Exposure Therapy via Reinforcement Learning

2025-04-18

AI Summary

Virtual reality exposure therapy (VRET) suffers from poor personalization and limited generalizability due to reliance on static, manually engineered rules for stimulus delivery. Method: This study proposes the Experience-Driven Procedural Content Generation with Reinforcement Learning (EDPCGRL) framework—a novel physiological-signal-driven adaptive approach that integrates experience-based procedural content generation with Proximal Policy Optimization (PPO) reinforcement learning to enable real-time, individualized virtual stimulus generation for arachnophobia VRET. Stimulus intensity is dynamically optimized using real-time physiological feedback—specifically heart rate variability—eliminating the need for hand-crafted rule encoding. Contribution/Results: A human-subject experiment demonstrates that EDPCGRL significantly improves anxiety regulation efficiency and treatment adherence compared to conventional rule-based methods (p < 0.01), while exhibiting robust cross-subject generalizability. This work establishes a new paradigm for automated, physiology-informed, and personalized psychological intervention.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Personalized therapy, in which a therapeutic practice is adapted to an individual patient, can lead to improved health outcomes. Typically, this is accomplished by relying on a therapist's training and intuition along with feedback from a patient. However, this requires the therapist to become an expert on any technological components, such as in the case of Virtual Reality Exposure Therapy (VRET). While there exist approaches to automatically adapt therapeutic content to a patient, they generally rely on hand-authored, pre-defined rules, which may not generalize to all individuals. In this paper, we propose an approach to automatically adapt therapeutic content to patients based on physiological measures. We implement our approach in the context of virtual reality arachnophobia exposure therapy, and rely on experience-driven procedural content generation via reinforcement learning (EDPCGRL) to generate virtual spiders to match an individual patient. Through a human subject study, we demonstrate that our system significantly outperforms a more common rules-based method, highlighting its potential for enhancing personalized therapeutic interventions.

Problem

• Automating personalized VR exposure therapy adaptation
• Replacing manual rules with RL-driven content generation
• Optimizing arachnophobia treatment via physiological feedback

Innovation

• Reinforcement learning for personalized therapy
• Physiological measures guide content adaptation
• Procedural content generation for VRET