Multi-Modality Sensing in mmWave Beamforming for Connected Vehicles Using Deep Learning

2025-04-08

IEEE Transactions on Cognitive Communications and Networking

AI Summary

In high-mobility vehicular networks, conventional beam scanning for millimeter-wave (mmWave) V2I/V2V communications incurs prohibitive computational and signaling overhead. To address this, we propose a proactive beam prediction method driven by multimodal perception data (e.g., radar and IMU). This is the first work to deeply integrate on-board heterogeneous sensor measurements with deep neural networks—eliminating reliance on real-time channel state information or exhaustive beam sweeping—enabling rapid line-of-sight beam alignment. Key innovations include spatiotemporal sensor synchronization, empirical mmWave channel modeling from real-world measurements, and end-to-end multimodal feature fusion. Evaluated on authentic vehicular datasets, our model achieves 98.19% Top-13 beam prediction accuracy, reducing beam search space and latency by 79.67% and 91.89%, respectively. The approach significantly enhances the practicality and efficiency of mmWave communications under high-speed mobility.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Large-scale security measurements

Abstract

Beamforming techniques are considered as essential parts to compensate severe path losses in millimeter-wave (mmWave) communications. In particular, these techniques adopt large antenna arrays and formulate narrow beams to obtain satisfactory received powers. However, performing accurate beam alignment over narrow beams for efficient link configuration by traditional standard defined beam selection approaches, which mainly rely on channel state information and beam sweeping through exhaustive searching, imposes computational and communications overheads. And, such resulting overheads limit their potential use in vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) communications involving highly dynamic scenarios. In comparison, utilizing out-of-band contextual information, such as sensing data obtained from sensor devices, provides a better alternative to reduce overheads. This paper presents a deep learning-based solution for utilizing the multi-modality sensing data for predicting the optimal beams having sufficient mmWave received powers so that the best V2I and V2V line-of-sight links can be ensured proactively. The proposed solution has been tested on real-world measured mmWave sensing and communication data, and the results show that it can achieve up to 98.19% accuracies while predicting top-13 beams. Correspondingly, when compared to existing been sweeping approach, the beam sweeping searching space and time overheads are greatly shortened roughly by 79.67% and 91.89%, respectively which confirm a promising solution for beamforming in mmWave enabled communications.

Problem

• Reducing beam alignment overheads in mmWave V2I/V2V communications
• Predicting optimal beams using multi-modality sensing data
• Improving beamforming efficiency with deep learning techniques

Innovation

• Deep learning predicts optimal mmWave beams
• Multi-modality sensing reduces beam alignment overheads
• Achieves high accuracy with reduced search space