SurgWMBench: A Vision-Based Benchmark for World-Modeling Surgical Instrument Motion Planning

2026-08-08

AI Summary

Existing methods for surgical video understanding struggle to jointly model the evolution of visual states and the dynamics of instrument actions, and lack evaluation benchmarks tailored for motion planning. This work proposes SurgWMBench—the first standardized benchmark specifically designed to assess the instrument motion planning capability within surgical visual world models—addressing the critical gap in publicly available datasets and evaluation protocols. Built upon intraoperative image sequences and historical trajectories, SurgWMBench quantitatively evaluates the feasibility and stability of predicted trajectories through metrics such as geometric accuracy, temporal consistency, and robustness. The framework supports performance analysis under continuous rollout or input perturbations, thereby advancing surgical planning systems toward prospective modeling.

Abstract

Reliable surgical planning requires models that move beyond recognizing the current surgical step or imitating expert demonstrations, and instead anticipate how instrument motion reshapes subsequent operative states. Most surgical video understanding methods focus on recognizing phases, actions, or workflow states, while providing limited support for explicitly modeling instrument motion. Conversely, existing tool motion prediction methods can forecast instrument trajectories, but they generally do not capture the coupled evolution of future surgical video states. World models offer a natural framework for jointly modeling visual state transitions and instrument motion dynamics. However, existing surgical world model studies remain largely centered on visual generation quality, relying on generation-oriented metrics such as FVD and CD-FVD. These metrics are poorly aligned with instrument motion planning, as they do not directly measure whether predicted trajectories are geometrically accurate, temporally coherent, or actionable for downstream planning. This limitation is partly structural, since the field lacks public datasets and standardized evaluation protocols that provide the benchmarking infrastructure needed to assess motion-centric capabilities in surgical world models. In this paper, we introduce SurgWMBench, a vision-based benchmark for short-horizon surgical motion planning and dynamics prediction. Given intraoperative image sequences and historical instrument trajectory, SurgWMBench evaluates both near-future instrument motion prediction and stability under continuous rollout or input perturbations.

Problem

• surgical world models
• instrument motion planning
• vision-based benchmark
• motion prediction
• evaluation protocol

Innovation

• surgical world model
• instrument motion planning
• vision-based benchmark
• trajectory prediction
• SurgWMBench