NeoLoc-68: End-to-end 68-point neonatal facial landmark localisation in neonatal clinical environments

2026-06-18

AI Summary

This study addresses the significant performance degradation of existing facial landmark detection models—trained on adult faces—when applied to neonatal clinical settings, where challenges such as device occlusion, highly variable head poses, and motion blur are prevalent. To overcome these limitations, this work proposes the first end-to-end 68-point facial landmark detection model specifically designed for newborns, built upon a YOLO-based keypoint regression architecture. The model leverages a pretrained neonatal face detector for initialization and is trained on a harmonized multi-source dataset combining 11 public benchmarks with in-house clinical data. It achieves state-of-the-art performance on public test sets (NME = 5.37%, FR = 12.5%) and, after fine-tuning on clinical data, further reduces the failure rate to 1.77% (DFR), substantially outperforming existing approaches and providing a critical enabler for contactless neonatal pain assessment.

Abstract

Facial landmark localisation is a prerequisite for developing automated, non-contact neonatal pain assessment methods. Clinicians use pain scales to judge the severity of pain, many of which rely on facial expression. However, facial landmark detectors trained on adult faces perform poorly in neonatal clinical environments due to frequent occlusions caused by medical equipment, varied head poses, and challenging imaging conditions, including motion blur triggered by sudden pain-related movements. We propose an end-to-end facial landmark detector capable of predicting 68 landmarks on neonatal faces in clinical environments. We combined 37,459 single-face images from 11 public datasets, standardised to 68-point markup, with 1,123 manually annotated frames from a neonatal research dataset (totalling over 76,000 landmarks). A YOLO-based keypoint model was adapted to regress the facial landmarks, initialised with weights from a pretrained neonatal face detector. On public datasets, our proposed model achieved state-of-the-art performance: Normalised Mean Error (NME) = 5.37, Failure Rate (FR) = 12.5%, Area Under the Cumulative Error Curve (AUC) at AUC0.08 = 38.00% and AUC0.1 = 48.70%. On the clinical neonatal test set, before fine-tuning, the model achieved the lowest Detection Failure Rate (DFR) = 5.3% among all baselines and showed strong generalisation. After fine-tuning, performance improved further to NME = 6.36, FR = 22.30%, DFR = 1.77%, AUC0.08 = 29.24% and AUC0.1 = 40.25%. To the best of our knowledge, this represents the first end-to-end 68-point neonatal facial landmark detection model. With further dataset expansion and refinement, it could support downstream tasks in neonatal health monitoring and pain-related facial analysis.

Problem

• neonatal facial landmark localisation
• occlusion
• clinical environment
• facial expression analysis
• pain assessment

Innovation

• neonatal facial landmark localisation
• end-to-end keypoint detection
• YOLO-based model
• clinical environment robustness
• 68-point annotation