CapCLIP: A Vision-Language Representation Alignment Approach for Wireless Capsule Endoscopy Analysis

2026-05-08

AI Summary

Wireless capsule endoscopy analysis faces significant challenges, including the vast volume of images, difficulty in anomaly detection, and the limited generalization and narrow pathological coverage of existing models. This work proposes the first vision–language representation alignment framework tailored to this domain, built upon the CLIP architecture and enhanced with clinical knowledge–driven standardized terminology and pathology-aware text templates to learn semantically rich and transferable multimodal embeddings. By introducing language-guided representation learning into capsule endoscopy for the first time, the method enables multi-task evaluation under a strict zero-shot setting. Experiments demonstrate consistent superiority over current baselines across k-nearest neighbor classification, image–text classification, and text-to-image retrieval tasks, with particularly notable improvements in zero-shot classification and cross-modal retrieval performance on out-of-distribution data.

Abstract

Wireless capsule endoscopy (WCE) enables non-invasive visual assessment of the small bowel, but its clinical utility is constrained by the large volume of frames generated per examination and the difficulty of recognising subtle abnormalities under highly variable imaging conditions. Existing learning-based approaches for WCE are predominantly vision-only, often confined to narrow pathology sets, and show limited transfer across datasets and centres. To address these limitations, this study introduces CapCLIP, a domain-specific vision-language representation learning framework for WCE. CapCLIP aligns capsule endoscopy frames with clinically grounded textual descriptions derived from standardised nomenclature and pathology-aware caption templates, thereby learning embeddings that are both semantically informed and transferable. The proposed framework is evaluated against relevant open-source vision and vision-language foundation models under strict zero-shot conditions using unseen WCE datasets. Evaluation covers three downstream tasks: K-nearest neighbour classification, CLIP-style image-text classification, and text-to-image retrieval. Across these settings, CapCLIP consistently outperforms the compared baselines, with particularly strong gains in zero-shot image-text classification and cross-modal retrieval on out-of-distribution datasets. The results indicate that language-guided representation learning can improve both generalisation and semantic interpretability in WCE analysis. These findings position CapCLIP as a step toward foundation models tailored to capsule endoscopy and support the use of language-grounded WCE analysis.

Problem

• Wireless Capsule Endoscopy
• vision-language representation
• zero-shot generalization
• cross-dataset transfer
• subtle abnormality detection

Innovation

• vision-language alignment
• wireless capsule endoscopy
• zero-shot learning
• foundation model
• cross-modal retrieval