TruckV2X: A Truck-Centered Perception Dataset

2025-07-13

AI Summary

Existing autonomous driving cooperative perception datasets predominantly focus on passenger vehicles, exhibiting a critical scarcity of large-scale, multi-agent, multimodal benchmarks tailored for heavy-duty trucks—hindering research on cooperative perception in scenarios characterized by extensive blind zones and complex trailer dynamics. To address this gap, we introduce TruckCoop, the first large-scale, truck-centric cooperative perception dataset. It comprehensively covers tractors, semi-trailers, connected autonomous vehicles, and roadside units, enabling the first V2X-driven, multi-agent cooperative perception modeling. TruckCoop integrates synchronized LiDAR and camera modalities and is accompanied by an occlusion-robust cooperative perception benchmark. Extensive experiments demonstrate that TruckCoop significantly improves performance in object detection and trajectory prediction under severe occlusion conditions. As a foundational infrastructure, it advances both algorithmic development and real-world deployment of cooperative perception systems for heavy-duty vehicles.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Multiagent Systems: Teamwork

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Autonomous trucking offers significant benefits, such as improved safety and reduced costs, but faces unique perception challenges due to trucks' large size and dynamic trailer movements. These challenges include extensive blind spots and occlusions that hinder the truck's perception and the capabilities of other road users. To address these limitations, cooperative perception emerges as a promising solution. However, existing datasets predominantly feature light vehicle interactions or lack multi-agent configurations for heavy-duty vehicle scenarios. To bridge this gap, we introduce TruckV2X, the first large-scale truck-centered cooperative perception dataset featuring multi-modal sensing (LiDAR and cameras) and multi-agent cooperation (tractors, trailers, CAVs, and RSUs). We further investigate how trucks influence collaborative perception needs, establishing performance benchmarks while suggesting research priorities for heavy vehicle perception. The dataset provides a foundation for developing cooperative perception systems with enhanced occlusion handling capabilities, and accelerates the deployment of multi-agent autonomous trucking systems. The TruckV2X dataset is available at https://huggingface.co/datasets/XieTenghu1/TruckV2X.

Problem

• Addresses truck perception challenges like blind spots and occlusions
• Lacks datasets for multi-agent heavy-duty vehicle cooperative perception
• Introduces TruckV2X dataset to enhance occlusion handling in autonomous trucking

Innovation

• First large-scale truck-centered cooperative perception dataset
• Multi-modal sensing with LiDAR and cameras
• Multi-agent cooperation including tractors and trailers