TCFNet: Bidirectional face-bone transformation via a Transformer-based coarse-to-fine point movement network

2025-08-19

AI Summary

In orthognathic surgery, simulating bone–facial point cloud shape transformations faces challenges including high computational cost, strong dependence on pre-/post-registration, loss of local geometric details, and limited scalability to large-scale point clouds. To address these issues, we propose a bidirectional coarse-to-fine Transformer architecture that jointly leverages global contextual modeling (via Transformer) and local geometric modeling (via LIA-Net). Our method incorporates edge, directional, and relative positional features, integrates a deformable-registration-inspired auxiliary loss, and employs gated recurrent units to refine critical organ reconstruction. Unlike existing deep learning approaches, ours eliminates complex registration preprocessing and postprocessing, significantly enhancing receptive field coverage and robustness to noise. Evaluated on a newly constructed clinical dataset, our method achieves superior quantitative metrics and visual fidelity over state-of-the-art methods, enabling high-accuracy, end-to-end, large-scale point cloud displacement prediction with direct clinical applicability.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Computer-aided surgical simulation is a critical component of orthognathic surgical planning, where accurately simulating face-bone shape transformations is significant. The traditional biomechanical simulation methods are limited by their computational time consumption levels, labor-intensive data processing strategies and low accuracy. Recently, deep learning-based simulation methods have been proposed to view this problem as a point-to-point transformation between skeletal and facial point clouds. However, these approaches cannot process large-scale points, have limited receptive fields that lead to noisy points, and employ complex preprocessing and postprocessing operations based on registration. These shortcomings limit the performance and widespread applicability of such methods. Therefore, we propose a Transformer-based coarse-to-fine point movement network (TCFNet) to learn unique, complicated correspondences at the patch and point levels for dense face-bone point cloud transformations. This end-to-end framework adopts a Transformer-based network and a local information aggregation network (LIA-Net) in the first and second stages, respectively, which reinforce each other to generate precise point movement paths. LIA-Net can effectively compensate for the neighborhood precision loss of the Transformer-based network by modeling local geometric structures (edges, orientations and relative position features). The previous global features are employed to guide the local displacement using a gated recurrent unit. Inspired by deformable medical image registration, we propose an auxiliary loss that can utilize expert knowledge for reconstructing critical organs.Compared with the existing state-of-the-art (SOTA) methods on gathered datasets, TCFNet achieves outstanding evaluation metrics and visualization results. The code is available at https://github.com/Runshi-Zhang/TCFNet.

Problem

• Accurately simulating face-bone transformations for surgical planning
• Overcoming computational inefficiency in biomechanical simulation methods
• Handling large-scale point clouds with precise local geometric structures

Innovation

• Transformer-based coarse-to-fine point movement network
• Local information aggregation network modeling geometric structures
• Auxiliary loss utilizing expert knowledge for reconstruction