Smartphone-based eye tracking system using edge intelligence and model optimisation

2024-08-22

Internet of Things

AI Summary

Existing mobile eye-tracking systems suffer from low accuracy and high latency in real-time interactive scenarios—such as video playback, gaming, and AR/VR—due to constraints in on-device computational resources, power consumption, and reliance on cloud or external hardware. This paper introduces the first lightweight, real-time eye-tracking system designed natively for smartphones, requiring no peripherals or cloud dependency. Our approach innovatively combines neural architecture search (NAS) with knowledge distillation for aggressive model compression; enhances YOLOv8 with a customized detection head; introduces an adaptive iris optical flow alignment module; and integrates TensorRT-based on-device acceleration with federated edge-coordinated calibration. Evaluated on iPhone 13 and Pixel 6, the system achieves a mean inference latency of 23 ms and an average gaze estimation error of 0.85°, while reducing model parameters by 90%, lowering latency by 75% versus prior work, and improving energy efficiency by 32% over state-of-the-art methods.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Problem

• Smartphone Eye Tracking
• Accuracy Improvement
• Interactive Content

Innovation

• Eye Tracking
• Neural Networks (CNN, LSTM, GRU)
• Mobile Optimization