A Survey on Deep Multi-Task Learning in Connected Autonomous Vehicles

2025-07-29

AI Summary

Concurrent execution of multiple perception and prediction tasks—e.g., object detection, semantic segmentation, depth estimation, trajectory forecasting, and behavior prediction—in connected autonomous vehicles (CAVs) leads to model redundancy, excessive computational overhead, and poor real-time performance. Method: This paper presents the first systematic survey of deep multi-task learning (MTL) in CAVs, covering perception, prediction, planning, control, and V2X cooperative intelligence. We propose a novel MTL taxonomy tailored to connected environments, integrating V2X communication with shared backbone networks to enable cross-task knowledge transfer and joint optimization. Contribution/Results: We identify critical limitations of existing approaches—including insufficient generalization to dynamic scenes, inadequate task conflict mitigation, and weak modeling of vehicle-infrastructure cooperation—and pinpoint three key research gaps. Finally, we outline future directions: scalable architecture design, uncertainty-aware joint training, and lightweight cooperative inference—providing both theoretical foundations and practical paradigms for efficient, real-time, and robust onboard AI.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Multiagent Systems: Multiagent Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Connected autonomous vehicles (CAVs) must simultaneously perform multiple tasks, such as object detection, semantic segmentation, depth estimation, trajectory prediction, motion prediction, and behaviour prediction, to ensure safe and reliable navigation in complex environments. Vehicle-to-everything (V2X) communication enables cooperative driving among CAVs, thereby mitigating the limitations of individual sensors, reducing occlusions, and improving perception over long distances. Traditionally, these tasks are addressed using distinct models, which leads to high deployment costs, increased computational overhead, and challenges in achieving real-time performance. Multi-task learning (MTL) has recently emerged as a promising solution that enables the joint learning of multiple tasks within a single unified model. This offers improved efficiency and resource utilization. To the best of our knowledge, this survey is the first comprehensive review focused on MTL in the context of CAVs. We begin with an overview of CAVs and MTL to provide foundational background. We then explore the application of MTL across key functional modules, including perception, prediction, planning, control, and multi-agent collaboration. Finally, we discuss the strengths and limitations of existing methods, identify key research gaps, and provide directions for future research aimed at advancing MTL methodologies for CAV systems.

Problem

• Addressing multiple tasks in CAVs using single unified models
• Reducing deployment costs and computational overhead in CAVs
• Improving real-time performance and efficiency in autonomous driving

Innovation

• Deep multi-task learning for connected autonomous vehicles
• Single unified model for multiple perception tasks
• V2X communication enhances cooperative driving efficiency