Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning

2026-06-20

AI Summary

This work addresses the challenges of target–peak association ambiguity and inseparability of co-resident targets within the same range–Doppler resolution cell in OFDM-based integrated sensing and communication (OFDM-ISAC) systems under multi-target scenarios. To overcome these limitations, the study introduces visual modality assistance for the first time by embedding street-view images into the OFDM waveform and fusing reconstructed image-based object detection outputs with delay–Doppler maps and geometric information through a multimodal framework. A high-dimensional classifier is trained using a novel triangle soft-label KL divergence loss, integrating DeepJSCC, YOLOv5, and the multimodal fusion network. Evaluated on a Blender-based vehicle simulation platform, the system achieves a localization RMSE of 16 cm and a delay RMSE of 10.8 ns; removing the visual modality degrades localization accuracy by up to 60×, demonstrating the method’s effectiveness and innovation in resolving ISAC data association and resolution bottlenecks.

Abstract

Orthogonal frequency division multiplexing (OFDM)-based integrated sensing and communication (ISAC) systems commonly extract target parameters by peak-searching a delay-Doppler map (DDM) constructed from reflected pilots. In multi-target scenarios, this results in ambiguity: the DDM does not reveal which physical target produced which peak, and two targets within the same delay-Doppler resolution cell cannot be separated. We propose a vision-assisted OFDM-ISAC framework that resolves both limitations by fusing wireless and visual modalities. The transmitter encodes an onboard street-view image with deep joint source-channel coding (DeepJSCC) and transmits it over the same OFDM waveform used for sensing; the receiver reconstructs the image, runs a fine-tuned YOLOv5 detector and fuses the resulting per-target features (bounding-box coordinates and class labels) with the DDM and transmitter-receiver geometry through a learned multi-modal network. To stabilize training of the high dimensional delay and Doppler classifiers, we introduce a Kullback Leibler loss against triangular soft labels centered on the ground-truth bin. On a Blender-rendered vehicular testbed, the proposed framework achieves a 16 cm localization root mean square error (RMSE) and a 10.8 ns delay RMSE. An ablation study confirms that removing the visual modality causes a 60x degradation in localization. These results highlight the potential of vision to overcome the data-association and resolution limits of single-modality ISAC.

Problem

• multi-target association
• OFDM-ISAC
• delay-Doppler ambiguity
• resolution limit
• data association

Innovation

• vision-aided ISAC
• multi-modal learning
• DeepJSCC
• delay-Doppler association
• data fusion