Enhancing Predictability of Multi-Tenant DNN Inference for Autonomous Vehicles'Perception

2026-02-11

AI Summary

This work addresses the challenge of balancing real-time performance, predictability, and accuracy in multi-tenant deep neural network (DNN) perception tasks under resource-constrained autonomous driving systems. To this end, the authors propose PP-DNN, a novel system that introduces, for the first time, an environment-aware dynamic selection mechanism for keyframes and regions of interest (ROIs). By leveraging inter-frame similarity and traffic scene analysis, PP-DNN cohesively orchestrates an ROI generator, a FLOPs predictor, an ROI scheduler, and a non-keyframe detector to drastically reduce redundant computation. Experimental results on the BDD100K and nuScenes datasets demonstrate that PP-DNN achieves a 7.3× increase in fused frames, over 2.6× lower latency, more than 2.3× reduced latency jitter, a 75.4% improvement in detection completeness, and up to 98% enhancement in cost efficiency.

Technology Category

• Computer Vision: Vision for Robotics & Autonomous Driving
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Autonomous vehicles (AVs) rely on sensors and deep neural networks (DNNs) to perceive their surrounding environment and make maneuver decisions in real time. However, achieving real-time DNN inference in the AV's perception pipeline is challenging due to the large gap between the computation requirement and the AV's limited resources. Most, if not all, of existing studies focus on optimizing the DNN inference time to achieve faster perception by compressing the DNN model with pruning and quantization. In contrast, we present a Predictable Perception system with DNNs (PP-DNN) that reduce the amount of image data to be processed while maintaining the same level of accuracy for multi-tenant DNNs by dynamically selecting critical frames and regions of interest (ROIs). PP-DNN is based on our key insight that critical frames and ROIs for AVs vary with the AV's surrounding environment. However, it is challenging to identify and use critical frames and ROIs in multi-tenant DNNs for predictable inference. Given image-frame streams, PP-DNN leverages an ROI generator to identify critical frames and ROIs based on the similarities of consecutive frames and traffic scenarios. PP-DNN then leverages a FLOPs predictor to predict multiply-accumulate operations (MACs) from the dynamic critical frames and ROIs. The ROI scheduler coordinates the processing of critical frames and ROIs with multiple DNN models. Finally, we design a detection predictor for the perception of non-critical frames. We have implemented PP-DNN in an ROS-based AV pipeline and evaluated it with the BDD100K and the nuScenes dataset. PP-DNN is observed to significantly enhance perception predictability, increasing the number of fusion frames by up to 7.3x, reducing the fusion delay by>2.6x and fusion-delay variations by>2.3x, improving detection completeness by 75.4% and the cost-effectiveness by up to 98% over the baseline.

Problem

• multi-tenant DNN inference
• autonomous vehicles
• perception predictability
• real-time inference
• resource constraints

Innovation

• predictable inference
• multi-tenant DNNs
• region of interest (ROI)
• critical frame selection
• perception pipeline