Region-based Content Enhancement for Efficient Video Analytics at the Edge

2024-07-24

arXiv.org

AI Summary

Existing video analytics systems suffer from high computational overhead and low throughput due to full-frame enhancement, hindering real-time execution on edge devices. To address this, we propose a region-level content enhancement framework tailored for edge computing. Our approach introduces a novel macroblock-level region importance predictor for fine-grained identification of salient regions; a region-aware sparse enhancement mechanism that applies lightweight CNN-based enhancement exclusively to high-value macroblocks; and a performance-profiler-driven heterogeneous resource scheduling planner that dynamically optimizes compute allocation across hardware units. Evaluated on five representative edge platforms, our framework achieves 10–19% higher accuracy and 2–3× higher throughput compared to frame-level enhancement baselines—demonstrating a significant improvement in the accuracy-efficiency trade-off for edge video analytics.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Video Understanding & Activity Analysis
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Cloud, edge and content delivery systems for the Web
• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Video analytics is widespread in various applications serving our society. Recent advances of content enhancement in video analytics offer significant benefits for the bandwidth saving and accuracy improvement. However, existing content-enhanced video analytics systems are excessively computationally expensive and provide extremely low throughput. In this paper, we present region-based content enhancement, that enhances only the important regions in videos, to improve analytical accuracy. Our system, RegenHance, enables high-accuracy and high-throughput video analytics at the edge by 1) a macroblock-based region importance predictor that identifies the important regions fast and precisely, 2) a region-aware enhancer that stitches sparsely distributed regions into dense tensors and enhances them efficiently, and 3) a profile-based execution planer that allocates appropriate resources for enhancement and analytics components. We prototype RegenHance on five heterogeneous edge devices. Experiments on two analytical tasks reveal that region-based enhancement improves the overall accuracy of 10-19% and achieves 2-3x throughput compared to the state-of-the-art frame-based enhancement methods.

Problem

• Enhances important video regions for accuracy
• Reduces computational cost in video analytics
• Increases throughput in edge-based video systems

Innovation

• Region-based content enhancement
• Macroblock-based region predictor
• Profile-based execution planner