AVS: A Computational and Hierarchical Storage System for Autonomous Vehicles

2025-11-19

AI Summary

Autonomous vehicles (AVs) generate up to 14 TB of heterogeneous sensor data daily, posing significant challenges for onboard storage systems in simultaneously satisfying real-time write throughput, efficient query support, and stringent resource constraints. To address this, we propose a compute-storage-coordinated tiered in-vehicle storage system. Our approach features: (1) modality-aware compression—applying differentiated compression strategies per sensor modality (e.g., images, LiDAR point clouds, IMU streams); (2) dynamic hot/cold data tiering with daily archival; and (3) a lightweight embedded metadata index. The system is implemented atop an SSD/HDD hybrid filesystem, enabling hardware-software co-optimization. Evaluated on real-world L4 AV trajectories, it achieves stable real-time ingestion (≥1.2 GB/s), millisecond-scale selective retrieval, and an average end-to-end storage compression ratio of 3.8×. These improvements substantially enhance data accessibility and storage energy efficiency for downstream applications.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Data Compression
• Computer Vision: Image and Video Retrieval

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Data transparency and provenance

Abstract

Autonomous vehicles (AVs) are evolving into mobile computing platforms, equipped with powerful processors and diverse sensors that generate massive heterogeneous data, for example 14 TB per day. Supporting emerging third-party applications calls for a general-purpose, queryable onboard storage system. Yet today's data loggers and storage stacks in vehicles fail to deliver efficient data storage and retrieval. This paper presents AVS, an Autonomous Vehicle Storage system that co-designs computation with a hierarchical layout: modality-aware reduction and compression, hot-cold tiering with daily archival, and a lightweight metadata layer for indexing. The design is grounded with system-level benchmarks on AV data that cover SSD and HDD filesystems and embedded indexing, and is validated on embedded hardware with real L4 autonomous driving traces. The prototype delivers predictable real-time ingest, fast selective retrieval, and substantial footprint reduction under modest resource budgets. The work also outlines observations and next steps toward more scalable and longer deployments to motivate storage as a first-class component in AV stacks.

Problem

• Developing efficient storage for autonomous vehicles' massive sensor data
• Enabling queryable onboard storage for third-party applications
• Optimizing hierarchical storage with compression and metadata indexing

Innovation

• Co-designs computation with hierarchical storage layout
• Uses modality-aware reduction and compression techniques
• Implements hot-cold tiering with lightweight metadata indexing