SurgFormer: Scalable Learning of Organ Deformation with Resection Support and Real-Time Inference

2026-03-06

AI Summary

This work addresses the challenge of achieving real-time soft tissue deformation and resection interaction in surgical simulation, where high-fidelity biomechanical solvers are often prohibitively expensive computationally. To this end, the authors propose SurgFormer, a multi-resolution gated Transformer model that learns from eXtended Finite Element Method (XFEM)-generated supervisory data on volumetric meshes to jointly predict nodal displacement fields under both standard deformations and topological changes such as tissue resection. Key innovations include the first unified framework for co-modeling conventional deformation and XFEM-supervised resection dynamics, along with learnable resection embeddings and node- and channel-level gating mechanisms that adaptively fuse local and global information. Experiments demonstrate that SurgFormer significantly outperforms baseline methods on gallbladder and appendix resection datasets, achieving near real-time inference while maintaining high accuracy, thus enabling its use in interactive surgical simulation.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Medical and Biological Imaging

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce SurgFormer, a multiresolution gated transformer for data driven soft tissue simulation on volumetric meshes. High fidelity biomechanical solvers are often too costly for interactive use, so we train SurgFormer on solver generated data to predict nodewise displacement fields at near real time rates. SurgFormer builds a fixed mesh hierarchy and applies repeated multibranch blocks that combine local message passing, coarse global self attention, and pointwise feedforward updates, fused by learned per node, per channel gates to adaptively integrate local and long range information while remaining scalable on large meshes. For cut conditioned simulation, resection information is encoded as a learned cut embedding and provided as an additional input, enabling a unified model for both standard deformation prediction and topology altering cases. We also introduce two surgical simulation datasets generated under a unified protocol with XFEM based supervision: a cholecystectomy resection dataset and an appendectomy manipulation and resection dataset with cut and uncut cases. To our knowledge, this is the first learned volumetric surrogate setting to study XFEM supervised cut conditioned deformation within the same volumetric pipeline as standard deformation prediction. Across diverse baselines, SurgFormer achieves strong accuracy with favorable efficiency, making it a practical backbone for both tasks. {Code, data, and project page: \href{https://mint-vu.github.io/SurgFormer/}{available here}}

Problem

• soft tissue simulation
• real-time inference
• organ deformation
• resection support
• volumetric meshes

Innovation

• SurgFormer
• gated transformer
• cut-conditioned deformation
• XFEM supervision
• volumetric mesh simulation