ColoDiff: Integrating Dynamic Consistency With Content Awareness for Colonoscopy Video Generation

2026-02-26

AI Summary

This work proposes ColoDiff, a novel framework for colonoscopy video generation that addresses temporal inconsistency and limited clinical attribute control arising from irregular bowel anatomy, diverse pathologies, and variations in imaging modalities. The method introduces a TimeStream module to decouple inter-frame temporal dependencies and model dynamic consistency, alongside a Content-Aware module for precise intra-frame clinical attribute control. Innovatively, it incorporates a cross-frame tokenization mechanism, learnable prototype embeddings, and a non-Markovian sampling strategy to overcome the coarse-grained guidance limitations of conventional diffusion models, reducing sampling steps by over 90%. Extensive experiments on three public datasets and an in-house hospital database demonstrate that the generated videos exhibit smooth dynamics and rich anatomical detail, achieving strong performance across downstream tasks including disease diagnosis, modality classification, bowel preparation scoring, and lesion segmentation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Multimodal Learning

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• Responsible Web: Consent frameworks and practices on the web
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Colonoscopy video generation delivers dynamic, information-rich data critical for diagnosing intestinal diseases, particularly in data-scarce scenarios. High-quality video generation demands temporal consistency and precise control over clinical attributes, but faces challenges from irregular intestinal structures, diverse disease representations, and various imaging modalities. To this end, we propose ColoDiff, a diffusion-based framework that generates dynamic-consistent and content-aware colonoscopy videos, aiming to alleviate data shortage and assist clinical analysis. At the inter-frame level, our TimeStream module decouples temporal dependency from video sequences through a cross-frame tokenization mechanism, enabling intricate dynamic modeling despite irregular intestinal structures. At the intra-frame level, our Content-Aware module incorporates noise-injected embeddings and learnable prototypes to realize precise control over clinical attributes, breaking through the coarse guidance of diffusion models. Additionally, ColoDiff employs a non-Markovian sampling strategy that cuts steps by over 90% for real-time generation. ColoDiff is evaluated across three public datasets and one hospital database, based on both generation metrics and downstream tasks including disease diagnosis, modality discrimination, bowel preparation scoring, and lesion segmentation. Extensive experiments show ColoDiff generates videos with smooth transitions and rich dynamics. ColoDiff presents an effort in controllable colonoscopy video generation, revealing the potential of synthetic videos in complementing authentic representation and mitigating data scarcity in clinical settings.

Problem

• colonoscopy video generation
• temporal consistency
• content awareness
• data scarcity
• clinical attribute control

Innovation

• diffusion model
• temporal consistency
• content-aware generation
• colonoscopy video synthesis
• non-Markovian sampling