Conditional Graph Neural Network for Predicting Soft Tissue Deformation and Forces

2025-07-07

AI Summary

Soft tissue’s high deformability in virtual environments complicates joint modeling of deformation and force feedback; existing approaches rely on explicit tissue segmentation, mesh generation, and stiffness parameter estimation, resulting in poor generalizability. This paper proposes a conditional graph neural network (cGNN) that directly predicts both deformation and contact forces synchronously from sparse surface tracking data—without requiring explicit geometric modeling or material parameterization. The model is pre-trained on mass-spring system simulation data and fine-tuned using real-world experimental data via transfer learning. Experimental evaluation demonstrates a mean deformation prediction error of 0.35 ± 0.03 mm and a mean absolute contact force prediction error of 0.37 ± 0.05 N, significantly improving accuracy and cross-scenario generalization over prior methods. The approach establishes a new, efficient, and robust paradigm for soft tissue response modeling in interactive applications such as surgical simulation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Learning with Manifolds
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Soft tissue simulation in virtual environments is becoming increasingly important for medical applications. However, the high deformability of soft tissue poses significant challenges. Existing methods rely on segmentation, meshing and estimation of stiffness properties of tissues. In addition, the integration of haptic feedback requires precise force estimation to enable a more immersive experience. We introduce a novel data-driven model, a conditional graph neural network (cGNN) to tackle this complexity. Our model takes surface points and the location of applied forces, and is specifically designed to predict the deformation of the points and the forces exerted on them. We trained our model on experimentally collected surface tracking data of a soft tissue phantom and used transfer learning to overcome the data scarcity by initially training it with mass-spring simulations and fine-tuning it with the experimental data. This approach improves the generalisation capability of the model and enables accurate predictions of tissue deformations and corresponding interaction forces. The results demonstrate that the model can predict deformations with a distance error of 0.35$pm$0.03 mm for deformations up to 30 mm and the force with an absolute error of 0.37$pm$0.05 N for forces up to 7.5 N. Our data-driven approach presents a promising solution to the intricate challenge of simulating soft tissues within virtual environments. Beyond its applicability in medical simulations, this approach holds the potential to benefit various fields where realistic soft tissue simulations are required.

Problem

• Predict soft tissue deformation in virtual medical environments
• Estimate interaction forces for haptic feedback integration
• Overcome data scarcity with transfer learning techniques

Innovation

• Uses conditional graph neural network (cGNN)
• Combines surface points and force locations
• Employs transfer learning for data scarcity