PoseGaze-AHP: A Knowledge-Based 3D Dataset for AI-Driven Ocular and Postural Diagnosis

2025-10-04

AI Summary

Existing datasets model head pose and eye movements independently, hindering AI-assisted diagnosis of oculogenic abnormal head postures (AHP). To address this, we propose the first methodology for constructing a synchronized 3D head–eye motion dataset specifically designed for AI-driven oculogenic AHP analysis. Our approach jointly models head pose and gaze direction—unprecedented in prior work—and introduces a hierarchical, large language model (LLM)-based prompting strategy to structurally extract clinical knowledge with 91.92% accuracy. Leveraging the Neural Head Avatar (NHA) framework, we synthesize 7,920 high-fidelity 3D-rendered images covering diverse ophthalmic conditions. This dataset constitutes the first publicly available, synchronized, and geometrically precise 3D benchmark for clinically compliant and accurate AI-assisted AHP diagnosis—filling a critical gap in both ophthalmology and vision-based human-computer interaction research.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Diagnosing ocular-induced abnormal head posture (AHP) requires a comprehensive analysis of both head pose and ocular movements. However, existing datasets focus on these aspects separately, limiting the development of integrated diagnostic approaches and restricting AI-driven advancements in AHP analysis. To address this gap, we introduce PoseGaze-AHP, a novel 3D dataset that synchronously captures head pose and gaze movement information for ocular-induced AHP assessment. Structured clinical data were extracted from medical literature using large language models (LLMs) through an iterative process with the Claude 3.5 Sonnet model, combining stepwise, hierarchical, and complex prompting strategies. The extracted records were systematically imputed and transformed into 3D representations using the Neural Head Avatar (NHA) framework. The dataset includes 7,920 images generated from two head textures, covering a broad spectrum of ocular conditions. The extraction method achieved an overall accuracy of 91.92%, demonstrating its reliability for clinical dataset construction. PoseGaze-AHP is the first publicly available resource tailored for AI-driven ocular-induced AHP diagnosis, supporting the development of accurate and privacy-compliant diagnostic tools.

Problem

• Existing datasets separate head pose and gaze data
• Lack integrated diagnostic approaches for abnormal head posture
• Need AI-driven tools for synchronized ocular-postural assessment

Innovation

• Synchronously captures head pose and gaze movement
• Uses LLMs to extract clinical data from literature
• Transforms records into 3D images with NHA framework