Divide-then-Diagnose: Weaving Clinician-Inspired Contexts for Ultra-Long Capsule Endoscopy Videos

2026-04-23

AI Summary

This work addresses the challenges of sparse diagnostic events, severe redundant-frame interference, and the absence of diagnosis-driven summarization in capsule endoscopy videos by introducing a novel diagnosis-driven video summarization task. Inspired by clinical physicians’ reading workflows, the authors propose the DiCE framework, which first efficiently selects candidate frames, then constructs diagnostic context via a Context Weaver module, and finally aggregates multi-frame evidence through an Evidence Converger to support accurate diagnosis. To facilitate this research direction, they also present VideoCAP—the first capsule endoscopy video dataset annotated with clinical reports. Experimental results demonstrate that DiCE substantially outperforms existing methods, producing concise summaries with high clinical credibility and achieving superior performance in analyzing ultra-long capsule endoscopy videos.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Natural Language Processing: Summarization

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Capsule endoscopy (CE) enables non-invasive gastrointestinal screening, but current CE research remains largely limited to frame-level classification and detection, leaving video-level analysis underexplored. To bridge this gap, we introduce and formally define a new task, diagnosis-driven CE video summarization, which requires extracting key evidence frames that covers clinically meaningful findings and making accurate diagnoses from those evidence frames. This setting is challenging because diagnostically relevant events are extremely sparse and can be overwhelmed by tens of thousands of redundant normal frames, while individual observations are often ambiguous due to motion blur, debris, specular highlights, and rapid viewpoint changes. To facilitate research in this direction, we introduce VideoCAP, the first CE dataset with diagnosis-driven annotations derived from real clinical reports. VideoCAP comprises 240 full-length videos and provides realistic supervision for both key evidence frame extraction and diagnosis. To address this task, we further propose DiCE, a clinician-inspired framework that mirrors the standard CE reading workflow. DiCE first performs efficient candidate screening over the raw video, then uses a Context Weaver to organize candidates into coherent diagnostic contexts that preserve distinct lesion events, and an Evidence Converger to aggregate multi-frame evidence within each context into robust clip-level judgments. Experiments show that DiCE consistently outperforms state-of-the-art methods, producing concise and clinically reliable diagnostic summaries. These results highlight diagnosis-driven contextual reasoning as a promising paradigm for ultra-long CE video summarization.

Problem

• capsule endoscopy
• video summarization
• diagnosis-driven
• ultra-long video
• clinical evidence extraction

Innovation

• diagnosis-driven summarization
• capsule endoscopy
• ultra-long video analysis
• clinical context modeling
• evidence aggregation