On the Robustness of Temporal Vision-Language Models for Surgical Endoscopy Videos

2026-08-14

AI Summary

This study addresses the insufficient robustness of temporal vision-language models against artifacts in endoscopic videos by proposing RobustEndoCLIP, a lightweight adaptation framework. Leveraging VeRA parameter-efficient fine-tuning and multimodal alignment techniques, this method enhances model resilience to visual disturbances. Furthermore, we introduce Endo-C6, the first corruption benchmark specifically designed for endoscopy, to enable systematic performance evaluation. Experimental results demonstrate that RobustEndoCLIP significantly improves both robustness and alignment accuracy under worst-case clinical artifact scenarios, comprehensively outperforming existing baselines. Consequently, this work establishes a reliable few-shot adaptation paradigm for intelligent endoscopic analysis, effectively bridging the gap between general-purpose vision-language models and specialized clinical applications requiring high reliability under challenging imaging conditions.

Abstract

Temporal vision-language models (TVLMs) offer a reusable, prompt-based interface for surgical video understanding, yet, their robustness under clinically realistic acquisition artifacts in endoscopy remains insufficiently characterized. In practice, degradations such as defocus, haze, motion blur, noise, cautery smoke, and packet loss introduce structured distribution shifts which may compromise video-text alignment. We study the robustness of temporal VLMs under such shifts caused by corruptions in clip frames. We introduce Endo-C6, a compact corruption benchmark of six endoscopy-realistic perturbations evaluated at a fixed high severity, and apply it to public Gastrointestinal (GI) endoscopy and laparoscopic cholecystectomy videos. Under a standardized prompt protocol, we benchmark 3 recent surgical TVLM baselines and analyze robustness in both mean and worst-case settings, spanning 294 dataset-level evaluations. Finally, we present RobustEndoCLIP, obtained by few-shot parameter-efficient tuning with VeRA, outperforming existing TVLM baselines. Our findings show that off-the-shelf TVLMs can exhibit severe worst-case collapse under endoscopy-specific corruptions, whereas lightweight few-shot adaptation can substantially improve corrupted performance and robustness without changing the prompt-based interface. We expect Endo-C6 to support standardized robustness reporting and promote more reliable clinical vision-language systems.

Problem

• Temporal Vision-Language Models
• Surgical Endoscopy
• Robustness
• Distribution Shifts
• Acquisition Artifacts

Innovation

• Endo-C6
• RobustEndoCLIP
• VeRA
• Temporal Vision-Language Models
• Surgical Endoscopy