facial landmark detection

Large illumination variations and diverse patient poses in surgical settings degrade facial landmark localization accuracy. Method: This study proposes a robot-arm-based controllable evaluation framework: under fixed surgical lighting and a human phantom, a robotic arm systematically varies the camera viewpoint to quantitatively assess MediaPipe’s landmark detection performance across large yaw (±60°) and pitch (±45°) angles. Contribution/Results: This work establishes the first standardized, multi-angle, illumination-invariant evaluation protocol under realistic surgical constraints, revealing MediaPipe’s robustness limits and landmark dispersion bottlenecks under extreme poses. Experiments show a 32.7% reduction in mean landmark localization error versus free-view baselines—particularly pronounced for profile and downward-facing poses. The framework provides a reproducible validation paradigm and empirical foundation for optimizing lightweight, intraoperative facial analysis algorithms.

In digital dentistry, automatic localization of dental anatomical landmarks—such as cusp tips, facial axis points, and gingival margins—faces three key challenges: limited annotated training samples, high inter-subject anatomical variability, and the difficulty of modeling unordered 3D point clouds. To address these, we propose a geometrically aware and anatomically interpretable end-to-end point-level regression framework. Our method introduces a lightweight Point Transformer v3 module that jointly encodes point-wise features and captures local geometric structures via graph-based modeling. Furthermore, we design a graph neural network–driven non-maximum suppression (NMS) mechanism to robustly identify and refine local peaks corresponding to landmark positions. Evaluated on the MICCAI 2024 3DTeethLand Challenge, our approach achieves state-of-the-art performance, reducing mean localization error by 21.3% and significantly improving cross-subject generalization. This work establishes a reliable, clinically applicable benchmark for digital orthodontics and prosthodontics.

Current medical image keypoint localization tools suffer from insufficient anatomical accuracy, limited model customizability, and poor clinical adaptability. To address these limitations, we present the first modular, extensible PyTorch-based open-source toolkit specifically designed for anatomical landmark localization. It supports both 2D/3D static and adaptive heatmap regression, integrates multi-format I/O, plug-and-play preprocessing pipelines, and flexible model architectures. Our core contribution is a medical imaging–specific framework that explicitly incorporates anatomical structural priors, enables cross-modal generalization, and ensures compatibility with clinical workflows—thereby bridging critical gaps left by generic pose estimation methods in anatomical precision, few-shot adaptation, and multi-center deployment. Experiments demonstrate significant improvements in localization accuracy and substantial reductions in algorithm development, evaluation, and transfer overhead. The toolkit has been successfully deployed across multiple clinical imaging analysis tasks.

Manual annotation of anatomical landmarks on 3D facial scans is time-consuming, expert-dependent, and hinders clinical deployment. To address this, we propose a fully automated deep learning framework for high-precision localization of 50 key anatomical landmarks. Our method integrates coarse alignment, ROI selection, and a novel Patch-Attention PointCNN—eliminating complex input representations and enabling end-to-end training. Crucially, we introduce patch-level attention into point cloud CNNs for the first time, significantly enhancing local geometric modeling and structural consistency. Evaluated on 214 healthy adult scans, the framework achieves a mean landmark error of 3.686 mm; on the FaceScape dataset, it yields point-wise and distance errors of 0.41 mm and 0.38 mm, respectively—matching inter-rater human reproducibility. It demonstrates strong cross-dataset and regional generalization, offering a reliable, clinically deployable solution for craniofacial analysis.

This work addresses two key challenges in facial landmark detection: insufficient geometric relationship modeling and degradation of low-level features. To this end, we propose a cascaded dual-branch Vision Transformer (ViT) architecture. The architecture jointly integrates a channel-partitioned ViT—capturing inter-channel semantic dependencies—and a spatial-partitioned ViT—modeling long-range geometric correlations via linear bases in heatmaps—augmented with long-range skip connections to explicitly preserve low-level image features and mitigate information attenuation induced by intermediate supervision. Adopting a heatmap regression paradigm with cascaded prediction, our method enhances localization accuracy. Extensive experiments demonstrate state-of-the-art performance on three major benchmarks—WFLW, COFW, and 300W—with significantly reduced mean errors. Results validate the effectiveness of the dual-path ViT collaboration and the feature fidelity mechanism.

This work addresses the limitations of existing facial landmark detection methods, which suffer from insufficient accuracy under occlusion and lack explicit per-landmark visibility prediction. We propose the first unified occlusion-aware framework for general humanoid faces—encompassing both real human faces and stylized characters—that jointly predicts 100 dense landmark coordinates and their individual visibility. Built upon a heatmap-based backbone, our approach integrates local evidence with cross-landmark contextual information and introduces pseudo-visibility labels derived from mask–heatmap overlap, combined with manual annotations for hybrid supervision. A novel occlusion-aware evaluation protocol and a new dataset are established to enable comprehensive benchmarking. Experiments demonstrate that our method significantly improves robustness under external occlusions and large pose variations, notably enhancing localization accuracy in occluded regions while maintaining high precision on visible landmarks.

Existing automated methods struggle to emulate the reasoning process orthodontists employ when localizing cephalometric landmarks based on anatomical structures and geometric rules. This work proposes a five-stage anatomy-guided initialization pipeline that, for the first time, encodes clinical anatomical knowledge as spatial attention priors and integrates them into an HRNet-W32 detector via confidence-weighted fusion. This inductive bias substantially enhances the model’s generalization capability. Evaluated on 1,502 multi-source cephalograms, the method achieves a mean radial error of 1.04 mm across 25 landmarks—representing a 15.4% improvement over the current state of the art—with 12 landmarks exhibiting sub-millimeter accuracy. Ablation studies confirm that the performance gain stems from improved anatomical plausibility rather than the addition of extra input channels.

This work addresses the limitations of conventional facial landmark detection methods, which require separate models for different “N-point” datasets and can only predict a fixed number of landmarks, lacking both universality and flexibility. To overcome this, the authors propose a unified dynamic framework that introduces Face Part-Anchored Landmark Positions (FPALP), a novel representation that models arbitrary landmarks as normalized positions along facial contours. By integrating query-based dynamic prediction with a cross-modal decoder, the framework enables end-to-end regression of any number of landmarks. It supports joint training across multiple datasets and allows on-demand inference of user-specified landmarks. Extensive experiments demonstrate that the method achieves state-of-the-art or comparable performance across multiple benchmarks, significantly enhancing model generalizability and deployment flexibility.

This study addresses the challenge of accurately inferring craniofacial skeletal landmarks—unobservable in CT—from the geometry of external soft tissue surfaces. Leveraging paired data of soft-tissue point clouds and skeletal landmarks derived from co-registered CT scans, the work formulates a coordinate-consistent surface-to-bone mapping task and introduces a hierarchical point cloud neural network for implicit landmark estimation. For the first time, the feasibility of this inference is rigorously validated under controlled confounding factors, including registration accuracy, scanning domain, and acquisition conditions, revealing the critical contribution of non-anterior geometric information. Evaluated on a hold-out cohort of 40 patients, the method achieves a mean radial error of 2.97 mm (3.03 mm for deep landmarks), demonstrating the presence of subject-specific signals that surpass population-average anatomical configurations.

This work addresses the limitations of conventional methods in 3D anatomical landmark detection—namely, low efficiency, poor generalization, and inadequate cross-species adaptability—by proposing LmPT, a unified framework based on a conditional point cloud Transformer. By incorporating a conditioning mechanism, LmPT flexibly accommodates diverse input types and, for the first time, enables joint modeling and transfer learning of homologous skeletal landmarks across species, such as humans and dogs. Experimental results on a newly curated dataset of human femurs and a newly annotated dataset of canine femurs demonstrate that LmPT achieves high accuracy while exhibiting strong cross-species generalization capabilities. The code and datasets are publicly released to facilitate further research.