SteROI-D: System Design and Mapping for Stereo Depth Inference on Regions of Interest

2025-02-13

AI Summary

To address the excessive energy consumption of full-frame stereo depth estimation in AR/VR devices—rendering it unsuitable for battery-constrained scenarios—this paper proposes a system-level low-power depth estimation framework. Methodologically, it jointly exploits spatial region-of-interest (ROI) and temporal sparsity, introducing a novel dynamic ROI system-level mapping mechanism that enables flexible scheduling across heterogeneous architectures; it further designs an ROI-aware dataflow, configurable heterogeneous compute units, and a 28 nm ASIC prototype. The core contribution lies in breaking the conventional fixed-ROI paradigm, achieving hardware-algorithm co-optimization for fine-grained, time-varying ROIs. Experimental results demonstrate that, compared to a baseline ASIC, the proposed system reduces total energy consumption by up to 4.35×, significantly elevating the upper bound of energy-efficiency optimization.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Hardware-aware ML

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Machine learning algorithms have enabled high quality stereo depth estimation to run on Augmented and Virtual Reality (AR/VR) devices. However, high energy consumption across the full image processing stack prevents stereo depth algorithms from running effectively on battery-limited devices. This paper introduces SteROI-D, a full stereo depth system paired with a mapping methodology. SteROI-D exploits Region-of-Interest (ROI) and temporal sparsity at the system level to save energy. SteROI-D's flexible and heterogeneous compute fabric supports diverse ROIs. Importantly, we introduce a systematic mapping methodology to effectively handle dynamic ROIs, thereby maximizing energy savings. Using these techniques, our 28nm prototype SteROI-D design achieves up to 4.35x reduction in total system energy compared to a baseline ASIC.

Problem

• Energy-efficient stereo depth inference
• System design for AR/VR devices
• Dynamic Region-of-Interest mapping

Innovation

• Region-of-Interest energy saving
• Systematic mapping for dynamic ROIs
• Heterogeneous compute fabric