A Multimodal Clinically Informed Coarse-to-Fine Framework for Longitudinal CT Registration in Proton Therapy

2026-04-14

AI Summary

This work addresses the challenge of insufficient accuracy in longitudinal CT image registration for proton therapy, where anatomical changes often compromise alignment precision, and existing methods struggle to balance speed with clinical utility. To overcome this, the authors propose a coarse-to-fine deformable registration framework that systematically integrates multimodal clinical priors—including CT images, target and organ-at-risk contours, dose distributions, and treatment plan text—for the first time. The architecture employs dual CNN encoders and a Transformer decoder, enhanced by anatomy- and risk-guided attention mechanisms alongside a foreground-aware optimization strategy. Evaluated on a large-scale dataset of 1,222 CT scan pairs, the method significantly outperforms current state-of-the-art approaches, achieving fast, robust, and clinically meaningful high-precision registration.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Proton therapy offers superior organ-at-risk sparing but is highly sensitive to anatomical changes, making accurate deformable image registration (DIR) across longitudinal CT scans essential. Conventional DIR methods are often too slow for emerging online adaptive workflows, while existing deep learning-based approaches are primarily designed for generic benchmarks and underutilize clinically relevant information beyond images. To address this gap, we propose a clinically scalable coarse-to-fine deformable registration framework that integrates multimodal information from the proton radiotherapy workflow to accommodate diverse clinical scenarios. The model employs dual CNN-based encoders for hierarchical feature extraction and a transformer-based decoder to progressively refine deformation fields. Beyond CT intensities, clinically critical priors, including target and organ-at-risk contours, dose distributions, and treatment planning text, are incorporated through anatomy- and risk-guided attention, text-conditioned feature modulation, and foreground-aware optimization, enabling anatomically focused and clinically informed deformation estimation. We evaluate the proposed framework on a large-scale proton therapy DIR dataset comprising 1,222 paired planning and repeat CT scans across multiple anatomical regions and disease types. Extensive experiments demonstrate consistent improvements over state-of-the-art methods, enabling fast and robust clinically meaningful registration.

Problem

• deformable image registration
• proton therapy
• longitudinal CT
• clinical priors
• multimodal information

Innovation

• multimodal registration
• clinically informed attention
• coarse-to-fine deformable registration
• proton therapy
• transformer-based DIR