Federated Reinforcement Learning for Runtime Optimization of AI Applications in Smart Eyewears

2025-08-24

AI Summary

Smart glasses face severe constraints in computational capacity, memory, and battery life; cloud-dependent offloading further compromises real-time performance and robustness in XR AI applications due to network volatility and server load fluctuations. To address these challenges, we propose a runtime-optimized federated reinforcement learning framework that unifies synchronous and asynchronous federation strategies with a dynamic model aggregation mechanism, enabling privacy-preserving collaborative training across heterogeneous wearable devices. Integrated with edge-assisted coordination and adaptive local inference scheduling, the framework achieves resource-aware decision-making. Experimental evaluation demonstrates a 42% reduction in agent performance variance, significantly enhancing response stability and system reliability—particularly in latency-critical tasks such as real-time object detection. Our approach establishes a lightweight, robust, and privacy-secure AI deployment paradigm tailored for resource-constrained wearables.

Technology Category

• Machine Learning: Hardware-aware ML
• Intelligent Robots: Embodied AI
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

Extended reality technologies are transforming fields such as healthcare, entertainment, and education, with Smart Eye-Wears (SEWs) and Artificial Intelligence (AI) playing a crucial role. However, SEWs face inherent limitations in computational power, memory, and battery life, while offloading computations to external servers is constrained by network conditions and server workload variability. To address these challenges, we propose a Federated Reinforcement Learning (FRL) framework, enabling multiple agents to train collaboratively while preserving data privacy. We implemented synchronous and asynchronous federation strategies, where models are aggregated either at fixed intervals or dynamically based on agent progress. Experimental results show that federated agents exhibit significantly lower performance variability, ensuring greater stability and reliability. These findings underscore the potential of FRL for applications requiring robust real-time AI processing, such as real-time object detection in SEWs.

Problem

• Optimizing AI application runtime on resource-limited smart eyewear
• Addressing computational and battery constraints via federated learning
• Reducing performance variability in real-time object detection tasks

Innovation

• Federated Reinforcement Learning for collaborative training
• Synchronous and asynchronous model aggregation strategies
• Reduced performance variability for stability