OccFace: Unified Occlusion-Aware Facial Landmark Detection with Per-Point Visibility

2026-02-11

AI Summary

This work addresses the limitations of existing facial landmark detection methods, which suffer from insufficient accuracy under occlusion and lack explicit per-landmark visibility prediction. We propose the first unified occlusion-aware framework for general humanoid faces—encompassing both real human faces and stylized characters—that jointly predicts 100 dense landmark coordinates and their individual visibility. Built upon a heatmap-based backbone, our approach integrates local evidence with cross-landmark contextual information and introduces pseudo-visibility labels derived from mask–heatmap overlap, combined with manual annotations for hybrid supervision. A novel occlusion-aware evaluation protocol and a new dataset are established to enable comprehensive benchmarking. Experiments demonstrate that our method significantly improves robustness under external occlusions and large pose variations, notably enhancing localization accuracy in occluded regions while maintaining high precision on visible landmarks.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Accurate facial landmark detection under occlusion remains challenging, especially for human-like faces with large appearance variation and rotation-driven self-occlusion. Existing detectors typically localize landmarks while handling occlusion implicitly, without predicting per-point visibility that downstream applications can benefits. We present OccFace, an occlusion-aware framework for universal human-like faces, including humans, stylized characters, and other non-human designs. OccFace adopts a unified dense 100-point layout and a heatmap-based backbone, and adds an occlusion module that jointly predicts landmark coordinates and per-point visibility by combining local evidence with cross-landmark context. Visibility supervision mixes manual labels with landmark-aware masking that derives pseudo visibility from mask-heatmap overlap. We also create an occlusion-aware evaluation suite reporting NME on visible vs. occluded landmarks and benchmarking visibility with Occ AP, F1@0.5, and ROC-AUC, together with a dataset annotated with 100-point landmarks and per-point visibility. Experiments show improved robustness under external occlusion and large head rotations, especially on occluded regions, while preserving accuracy on visible landmarks.

Problem

• facial landmark detection
• occlusion
• per-point visibility
• human-like faces
• self-occlusion

Innovation

• occlusion-aware
• facial landmark detection
• per-point visibility
• heatmap-based localization
• visibility supervision