Sum-of-Checks: Structured Reasoning for Surgical Safety with Large Vision-Language Models

2026-04-23

AI Summary

This work addresses the lack of auditability and reliability in large vision-language models when assessing the Critical View of Safety (CVS) during laparoscopic cholecystectomy. To this end, the authors propose Sum-of-Checks, a structured reasoning framework that decomposes the CVS criteria into expert-defined binary clinical checks. The model evaluates each check individually, generates supporting evidence, and then aggregates the results via weighted scoring to produce a final assessment—explicitly separating evidence extraction from judgment for the first time in surgical decision-making. Experiments on the Endoscapes2023 benchmark demonstrate that this approach improves mean average precision (mAP) at the frame level by 12%–14% across three state-of-the-art models, significantly outperforming existing paradigms such as direct prompting and chain-of-thought reasoning, thereby enhancing both accuracy and interpretability of AI systems in surgical safety evaluation.

Technology Category

• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Purpose: Accurate assessment of the Critical View of Safety (CVS) during laparoscopic cholecystectomy is essential to prevent bile duct injury, a complication associated with significant morbidity and mortality. While large vision-language models (LVLMs) offer flexible reasoning, their predictions remain difficult to audit and unreliable on safety-critical surgical tasks. Methods: We introduce Sum-of-Checks, a framework that decomposes each CVS criterion into expert-defined reasoning checks reflecting clinically relevant visual evidence. Given a laparoscopic frame, an LVLM evaluates each check, producing a binary judgment and justification. Criterion-level scores are computed via fixed, weighted aggregation of check outcomes. We evaluate on the Endoscapes2023 benchmark using three frontier LVLMs, comparing against direct prompting, chain-of-thought, and sub-question decomposition, each with and without few-shot examples. Results: Sum-of-Checks improves average frame-level mean average precision by 12--14% relative to the best baseline across all three models and criteria. Analysis of individual checks reveals that LVLMs are reliable on observational checks (e.g., visibility, tool obstruction) but show substantial variability on decision-critical anatomical evidence. Conclusion: Structuring surgical reasoning into expert-aligned verification checks improves both accuracy and transparency of LVLM-based CVS assessment, demonstrating that explicitly separating evidence elicitation from decision-making is critical for reliable and auditable surgical AI systems. Code is available at https://github.com/BrachioLab/SumOfChecks.

Problem

• Critical View of Safety
• large vision-language models
• surgical safety
• bile duct injury
• auditable AI

Innovation

• Sum-of-Checks
• structured reasoning
• large vision-language models
• surgical safety
• Critical View of Safety