DepthPilot: From Controllability to Interpretability in Colonoscopy Video Generation

2026-04-28

AI Summary

Existing controllable medical video generation methods often lack anatomical consistency and clinical interpretability. To address this, this work proposes DepthPilot, a novel framework that, for the first time, integrates depth prior alignment and learnable spline functions into diffusion models. By incorporating geometric constraints and an adaptive spline-based denoising module, DepthPilot enables explicit geometry-guided nonlinear dynamic modeling. The approach requires only parameter-efficient fine-tuning to generate high-fidelity colonoscopy videos with accurate anatomical structures. Evaluated on multiple public and clinical datasets, DepthPilot achieves FID scores below 15, ranks first in physician assessments, and supports reliable 3D reconstruction and blind-spot identification—advancing medical video generation from mere controllability toward genuine clinical interpretability.

Abstract

Controllable medical video generation has achieved remarkable progress, but it still lacks interpretability, which requires the alignment of generated contents with physical priors and faithful clinical manifestations. To push the boundaries from mere controllability to interpretability, we propose DepthPilot, the first interpretable framework for colonoscopy video generation. This work takes a step toward trustworthy generation through two synergistic paradigms. To achieve explicit geometric grounding, DepthPilot devises a prior distribution alignment strategy, injecting depth constraints into the diffusion backbone via parameter-efficient fine-tuning to ensure anatomical fidelity. To enhance intrinsic nonlinear modeling under these geometric constraints, DepthPilot employs an adaptive spline denoising module, replacing fixed linear weights with learnable spline functions to capture complex spatio-temporal dynamics. Extensive evaluations across three public datasets and in-house clinical data confirm DepthPilot's robust ability to produce physically consistent videos. It achieves FID scores below 15 across all benchmarks and ranks first in clinician assessments, bridging the gap between "visually realistic" and "clinically interpretable". Moreover, DepthPilot-generated videos are expected to enable reliable 3D reconstruction, facilitating surgical navigation and blind region identification, and serve as a foundation toward the colorectal world model.

Problem

• controllable medical video generation
• interpretability
• physical priors
• clinical manifestations
• colonoscopy video

Innovation

• interpretable generation
• depth-constrained diffusion
• adaptive spline denoising
• geometric grounding
• colonoscopy video synthesis