SurgVLA-Bench: Towards Evaluating Vision-Language-Action Models for Laparoscopic Surgical Robotics

2026-06-28

AI Summary

This study addresses the absence of a standardized evaluation platform for vision–language–action (VLA) models in the context of laparoscopic surgical robotics. The authors introduce the first VLA benchmark specifically designed for this domain, leveraging the SurRoL simulation environment to construct a hierarchical task suite ranging from atomic actions to complete surgical procedures. They propose a multidimensional evaluation framework that jointly assesses action accuracy and semantic consistency. Experimental comparisons between autoregressive models (e.g., OpenVLA) and flow-matching models (e.g., π₀, SmolVLA) reveal that the former exhibit stronger semantic understanding, while the latter achieve higher task accuracy but suffer from limited generalization. The work further highlights how physical constraints inherent to endoscopic surgery—such as narrow field of view, restricted perspectives, and frequent occlusions—critically impede model performance.

Abstract

Vision-Language-Action (VLA) models represent a promising direction for embodied intelligence in surgical robotics. Despite the prevalence of VLA benchmarks for general robotics, standardized evaluation platforms specifically designed for surgical contexts remain absent. To address this limitation, we present SurgVLA-Bench, the first comprehensive benchmark for evaluating VLA models in laparoscopic surgical robotics. Leveraging the SurRoL simulation platform, we construct a hierarchical task taxonomy ranging from atomic actions to complete surgical procedures, complemented by a multi-dimensional evaluation framework assessing action accuracy and semantic consistency. We then systematically evaluate two representative paradigms, including autoregressive models such as OpenVLA, and flow matching models such as $π_{0}$, $π_{0.5}$, and SmolVLA. Our experiments show that autoregressive models tend to excel in semantic understanding, while flow matching models often achieve higher task precision but may face generalization trade-offs. However, even the best-performing models remain far from satisfactory, as the constrained endoscopic field of view, restricted viewing angles, and frequent occlusions persist as fundamental physical bottlenecks. The code and data are available at https://github.com/VCL-HNU/SurgVLA

Problem

• Vision-Language-Action models
• surgical robotics
• evaluation benchmark
• laparoscopic surgery
• embodied intelligence

Innovation

• Vision-Language-Action models
• surgical robotics benchmark
• laparoscopic surgery
• hierarchical task taxonomy
• multi-dimensional evaluation