DriveNetBench: An Affordable and Configurable Single-Camera Benchmarking System for Autonomous Driving Networks

2025-05-03

AI Summary

Neural network verification for autonomous driving typically relies on expensive hardware, hindering its adoption in educational settings and resource-constrained research environments. Method: This paper proposes a low-cost, configurable benchmarking system based on a single camera. It employs off-the-shelf, low-power hardware—including a Raspberry Pi and CSI camera—and implements a modular Python software stack supporting real-time inference for ONNX and TFLite models, integrated with joint evaluation tools for frame rate and accuracy (e.g., mAP). Contribution/Results: The work introduces the first open-source, lightweight, and reproducible benchmarking framework that eliminates dependence on specialized data acquisition equipment, enabling controlled, repeatable performance measurement under real-world driving conditions. Experiments demonstrate millisecond-level inference latency and consistent mAP evaluation across representative vision models, reducing benchmarking costs by 90%. The system has been successfully deployed and validated in multiple universities and startup teams.

Technology Category

• Computer Vision: Vision for Robotics & Autonomous Driving
• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Validating autonomous driving neural networks often demands expensive equipment and complex setups, limiting accessibility for researchers and educators. We introduce DriveNetBench, an affordable and configurable benchmarking system designed to evaluate autonomous driving networks using a single-camera setup. Leveraging low-cost, off-the-shelf hardware, and a flexible software stack, DriveNetBench enables easy integration of various driving models, such as object detection and lane following, while ensuring standardized evaluation in real-world scenarios. Our system replicates common driving conditions and provides consistent, repeatable metrics for comparing network performance. Through preliminary experiments with representative vision models, we illustrate how DriveNetBench effectively measures inference speed and accuracy within a controlled test environment. The key contributions of this work include its affordability, its replicability through open-source software, and its seamless integration into existing workflows, making autonomous vehicle research more accessible.

Problem

• Affordable benchmarking for autonomous driving networks
• Single-camera setup for standardized evaluation
• Replicable metrics for comparing network performance

Innovation

• Affordable single-camera benchmarking system
• Flexible software stack for model integration
• Standardized evaluation in real-world scenarios