Patient-Specific Deep Reinforcement Learning for Automatic Replanning in Head-and-Neck Cancer Proton Therapy

2025-06-11

AI Summary

In head-and-neck cancer proton therapy, anatomical changes frequently cause Bragg peak displacement, resulting in target underdosage and excessive organ-at-risk (OAR) irradiation; manual re-planning is time-consuming and labor-intensive. This work proposes the first patient-specific deep reinforcement learning (DRL) framework for automated intensity-modulated proton therapy (IMPT) re-planning: a personalized agent is trained on the patient’s baseline CT and augmented anatomical sequence; a DVH-driven state representation and a 22-dimensional prioritized action space are designed; and a 150-point multi-objective reward function jointly optimizes target coverage and OAR sparing. Evaluating with DQN and PPO algorithms on five patients, the PPO-based re-plans achieved a mean quality score of 142.74 ± 5.16—significantly higher than manual re-plans (137.20 ± 5.58, *p* < 0.05). Clinical validation confirmed improved target coverage and statistically significant OAR dose reduction.

Technology Category

• Machine Learning: Reinforcement Learning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Anatomical changes during intensity-modulated proton therapy (IMPT) for head-and-neck cancer (HNC) can shift Bragg peaks, risking tumor underdosing and organ-at-risk overdosing. As a result, treatment replanning is often required to maintain clinically acceptable treatment quality. However, current manual replanning processes are resource-intensive and time-consuming. We propose a patient-specific deep reinforcement learning (DRL) framework for automated IMPT replanning, with a reward-shaping mechanism based on a $150$-point plan quality score addressing competing clinical objectives. We formulate the planning process as an RL problem where agents learn control policies to adjust optimization priorities, maximizing plan quality. Unlike population-based approaches, our framework trains personalized agents for each patient using their planning CT (Computed Tomography) and augmented anatomies simulating anatomical changes (tumor progression and regression). This patient-specific approach leverages anatomical similarities throughout treatment, enabling effective plan adaptation. We implemented two DRL algorithms, Deep Q-Network and Proximal Policy Optimization, using dose-volume histograms (DVHs) as state representations and a $22$-dimensional action space of priority adjustments. Evaluation on five HNC patients using actual replanning CT data showed both DRL agents improved initial plan scores from $120.63 pm 21.40$ to $139.78 pm 6.84$ (DQN) and $142.74 pm 5.16$ (PPO), surpassing manual replans generated by a human planner ($137.20 pm 5.58$). Clinical validation confirms that improvements translate to better tumor coverage and OAR sparing across diverse anatomical changes. This work demonstrates DRL's potential in addressing geometric and dosimetric complexities of adaptive proton therapy, offering efficient offline adaptation solutions and advancing online adaptive proton therapy.

Problem

• Automates replanning for head-and-neck proton therapy
• Addresses anatomical changes causing dose distribution issues
• Reduces resource-intensive manual replanning with patient-specific DRL

Innovation

• Patient-specific DRL for automated IMPT replanning
• Reward-shaping with 150-point plan quality score
• Personalized agents using CT and augmented anatomies