🤖 AI Summary
This work addresses the challenge of efficiently executing multimodal perception tasks on low-power edge devices, where existing intelligent surveillance systems struggle with both computational efficiency and context-aware resource management. We propose a real-time multimodal vision framework tailored for the Raspberry Pi 5, integrating YOLOv8n for object detection, a customized FaceNet module for face recognition, and DeepFace for emotion classification. A context-triggered adaptive runtime scheduler dynamically activates subtasks only when needed, enabling effective task coordination while substantially reducing computational load. Experimental results demonstrate a 65% reduction in computational overhead, with an object detection AP of 0.861, 88% face recognition accuracy, and an emotion classification AUC up to 0.97, achieving an overall inference speed of 5.6 FPS. These findings validate the feasibility of deploying complex multimodal AI pipelines efficiently on cost-constrained edge hardware.
📝 Abstract
Intelligent surveillance systems often handle perceptual tasks such as object detection, facial recognition, and emotion analysis independently, but they lack a unified, adaptive runtime scheduler that dynamically allocates computational resources based on contextual triggers. This limits their holistic understanding and efficiency on low-power edge devices. To address this, we present a real-time multi-modal vision framework that integrates object detection, owner-specific face recognition, and emotion detection into a unified pipeline deployed on a Raspberry Pi 5 edge platform. The core of our system is an adaptive scheduling mechanism that reduces computational load by 65\% compared to continuous processing by selectively activating modules such as, YOLOv8n for object detection, a custom FaceNet-based embedding system for facial recognition, and DeepFace's CNN for emotion classification. Experimental results demonstrate the system's efficacy, with the object detection module achieving an Average Precision (AP) of 0.861, facial recognition attaining 88\% accuracy, and emotion detection showing strong discriminatory power (AUC up to 0.97 for specific emotions), while operating at 5.6 frames per second. Our work demonstrates that context-aware scheduling is the key to unlocking complex multi-modal AI on cost-effective edge hardware, making intelligent perception more accessible and privacy-preserving.