Training LLMs with Reinforcement Learning over Digital Twin Representations for Reasoning-Intensive Surgical VideoQA

2026-06-15

AI Summary

This work addresses the limitations of existing surgical video question-answering methods, which couple visual perception with multi-step reasoning and thereby disrupt spatiotemporal continuity, hindering complex reasoning capabilities. To overcome this, the authors propose a reinforcement learning framework that decouples perception from reasoning: a surgical foundation model constructs multi-level digital twin representations with uncertainty estimation, while a large language model is trained to perform hierarchical spatiotemporal reasoning. The approach introduces an innovative reward mechanism that integrates format validation, clinical plausibility, and uncertainty calibration. Evaluated on the newly introduced REAL-Colon-Reason benchmark as well as REAL-Colon-VQA and EndoVis18-VQA, the method achieves state-of-the-art performance across all datasets.

Abstract

Surgical video question answering requires multi-step reasoning across semantic, spatial, and temporal dimensions. Existing methods architecturally compress videos into discrete token representations and couple visual perception with reasoning. This approach fragments continuous spatial-temporal relationships and has been shown to restrict multi-step reasoning capabilities. We introduce a reinforcement learning (RL) framework that trains large language models (LLMs) to decouple perception from reasoning by operating over digital twin representations constructed from surgical foundation models. Additionally, we introduce hierarchical representations across frame, temporal window, and procedure levels with probabilistic uncertainty estimates. Finally, we propose a novel reward that combines format validation with accuracy assessment through clinical plausibility evaluation and uncertainty-aware calibration for training. To demonstrate the capabilities of this approach, we introduce REAL-Colon-Reason, a colonoscopic benchmark with 2000 question-answer pairs across three complexity levels. We achieve state-of-the-art performance on REAL-Colon-Reason and two existing surgical VideoQA benchmarks REAL-Colon-VQA and EndoVis18-VQA.

Problem

• Surgical VideoQA
• multi-step reasoning
• spatial-temporal relationships
• perception-reasoning coupling
• digital twin representations

Innovation

• Digital Twin
• Reinforcement Learning
• Surgical VideoQA
• Hierarchical Representation
• Uncertainty-aware Calibration