ECORE: Energy-Conscious Optimized Routing for Deep Learning Models at the Edge

2025-07-08

AI Summary

To address the challenge of jointly optimizing energy consumption and detection accuracy for real-time visual analytics on resource-constrained edge devices, this paper proposes a multi-strategy dynamic routing framework. The framework achieves context-aware, adaptive scheduling of image inference requests across heterogeneous edge hardware (Jetson Orin Nano, Raspberry Pi 4/5, TPU) and lightweight detection models (YOLO, SSD, EfficientDet) by integrating target-feature estimation with a greedy selection mechanism. It is the first to enable dynamic trade-offs between energy efficiency and accuracy based on target-scale and target-density characteristics. Experimental results demonstrate that, compared to an accuracy-first baseline, our approach reduces energy consumption by 45% and end-to-end latency by 49%, while incurring only a 2% mAP degradation. This significantly enhances both inference efficiency and practical deployability of real-time vision analytics at the edge.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Edge computing enables data processing closer to the source, significantly reducing latency an essential requirement for real-time vision-based analytics such as object detection in surveillance and smart city environments. However, these tasks place substantial demands on resource constrained edge devices, making the joint optimization of energy consumption and detection accuracy critical. To address this challenge, we propose ECORE, a framework that integrates multiple dynamic routing strategies including estimation based techniques and a greedy selection algorithm to direct image processing requests to the most suitable edge device-model pair. ECORE dynamically balances energy efficiency and detection performance based on object characteristics. We evaluate our approach through extensive experiments on real-world datasets, comparing the proposed routers against widely used baseline techniques. The evaluation leverages established object detection models (YOLO, SSD, EfficientDet) and diverse edge platforms, including Jetson Orin Nano, Raspberry Pi 4 and 5, and TPU accelerators. Results demonstrate that our proposed context-aware routing strategies can reduce energy consumption and latency by 45% and 49%, respectively, while incurring only a 2% loss in detection accuracy compared to accuracy-centric methods.

Problem

• Optimizing energy and accuracy for edge-based object detection
• Dynamic routing for efficient edge device-model pairing
• Reducing latency and energy in real-time vision analytics

Innovation

• Dynamic routing for edge device-model pairs
• Balances energy efficiency and detection performance
• Reduces energy and latency with minimal accuracy loss