Facial Surgery Preview Based on the Orthognathic Treatment Prediction

2024-12-15

arXiv.org

AI Summary

In orthognathic surgery consultation, existing visualization methods suffer from data scarcity and modeling complexity, limiting accurate postoperative facial appearance prediction. This paper proposes the first fully automatic 3D facial surgical preview system, generating high-fidelity postoperative facial predictions solely from preoperative imaging—without requiring additional medical scans. Methodologically: (1) we introduce a novel aesthetic loss function integrating oral convexity and asymmetry metrics; (2) we develop a clinically guided parametric reconstruction model coupled with a medically constrained latent code optimization mechanism; and (3) we design a tailored data augmentation strategy for small-sample orthognathic datasets. The system integrates the FLAME face model, a latent encoding network, multi-objective anatomical-aesthetic losses, and dense geometric correspondence modeling. Quantitative evaluation significantly outperforms baselines; qualitative results demonstrate precise facial contours and fine-grained details; and user studies confirm that both clinicians and laypersons cannot reliably distinguish predicted renderings from real postoperative images.

Technology Category

• Computer Vision: 3D Computer Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Orthognathic surgery consultation is essential to help patients understand the changes to their facial appearance after surgery. However, current visualization methods are often inefficient and inaccurate due to limited pre- and post-treatment data and the complexity of the treatment. To overcome these challenges, this study aims to develop a fully automated pipeline that generates accurate and efficient 3D previews of postsurgical facial appearances for patients with orthognathic treatment without requiring additional medical images. The study introduces novel aesthetic losses, such as mouth-convexity and asymmetry losses, to improve the accuracy of facial surgery prediction. Additionally, it proposes a specialized parametric model for 3D reconstruction of the patient, medical-related losses to guide latent code prediction network optimization, and a data augmentation scheme to address insufficient data. The study additionally employs FLAME, a parametric model, to enhance the quality of facial appearance previews by extracting facial latent codes and establishing dense correspondences between pre- and post-surgery geometries. Quantitative comparisons showed the algorithm's effectiveness, and qualitative results highlighted accurate facial contour and detail predictions. A user study confirmed that doctors and the public could not distinguish between machine learning predictions and actual postoperative results. This study aims to offer a practical, effective solution for orthognathic surgery consultations, benefiting doctors and patients.

Problem

• Develop automated 3D previews for orthognathic surgery outcomes
• Improve prediction accuracy with novel aesthetic and medical losses
• Address data insufficiency via augmentation and parametric modeling

Innovation

• Automated 3D preview pipeline for orthognathic surgery
• Novel aesthetic losses improve prediction accuracy
• FLAME model enhances facial appearance previews