Learning Where to Look: A Reinforcement Learning Framework for Robust Micro-Ultrasound Prostate Cancer Detection

2026-06-29

AI Summary

This study addresses key challenges in micro-ultrasound-based prostate cancer detection, including sparse annotations, high noise levels, class imbalance, and substantial inter-observer variability. The authors propose Prost-RL, a novel framework that reframes the task as a policy-driven, spatially aware reasoning problem. By integrating a lightweight reinforcement learning module, the model learns—prior to decoding—where to attend, generating interpretable spatial attention maps to guide cancer heatmap prediction and classification. The approach combines a foundation model encoder-decoder architecture with adaptive policy optimization (APO) and a noise-robust loss function, enabling accurate localization and transparent decision-making under weak supervision. Evaluated on 6,607 biopsy cores from 693 patients, the method achieves a core-level detection AUROC of 79.0 ± 3.5% and a sensitivity of 64.6 ± 6.3% at 80% specificity, with an AUROC of 79.3 ± 5.8% for clinically significant cancer classification—significantly outperforming baselines. Notably, the learned attention regions align closely with actual biopsy locations, offering clinically interpretable evidence.

Abstract

Micro-ultrasound ($μ$US) is a new, emerging, and promising imaging modality for prostate cancer (PCa) detection, but accurate identification of suspicious tissue remains highly dependent on clinical experience, leading to substantial inter-observer variability. Machine-learning assistance can reduce this variability; however, training reliable deep models is challenging because supervision is sparse and noisy -- typically limited to core-level histopathology outcomes (e.g., cancer grade and its percentage in a biopsy core) without pixel-level lesion annotations and under severe class imbalance. We introduce Prost-RL, which reframes $μ$US PCa detection as a spatially aware, policy-driven inference problem by learning where to look before decoding. Prost-RL integrates a lightweight reinforcement-learning policy into a foundation-model encoder-decoder to generate interpretable spatial attention maps that act as soft prompts for both cancer-likelihood heatmap prediction and image-level classification. We further propose Adaptive Policy Optimization (APO) to stabilize hybrid supervised-RL training and a noise-robust objective combining symmetric cross-entropy with negative-entropy regularization to mitigate weak-label noise and encourage sharp localization. On a cohort of 6,607 biopsy cores from 693 patients across five clinical sites, Prost-RL achieves $79.0\pm3.5$ AUROC with $64.6\pm6.3$% sensitivity at 80% specificity for core-level detection (+2.1 AUROC and +4.5 sensitivity points over the strongest baseline), and $79.3\pm5.8$ AUROC for clinically significant cancer classification. The learned policy highlights biopsy-aligned regions, providing transparent, spatially grounded evidence alongside quantitative risk predictions. Code is available at: https://github.com/DeepRCL/Prost-RL.

Problem

• micro-ultrasound
• prostate cancer detection
• weak supervision
• class imbalance
• inter-observer variability

Innovation

• reinforcement learning
• micro-ultrasound
• spatial attention
• weakly supervised learning
• prostate cancer detection