A Camera-Cooperative ISAC Framework for Multimodal Non-Cooperative UAVs Sensing

2026-05-21

AI Summary

This work addresses the limitations of single-modality sensing and the competition for integrated sensing and communication (ISAC) resources in non-cooperative drone detection by proposing a multimodal perception framework that synergistically combines camera and ISAC systems. The framework establishes a complementary perception loop wherein coarse-grained visual monitoring guides high-precision ISAC beam tracking. It introduces a novel vision-to-echo domain alignment mechanism (V2EDA), a cross-attention fusion architecture, and a multimodal state estimation algorithm (MMFE) to effectively mitigate resource contention and enhance sensing efficiency. Experimental results on the DeepSense 6G dataset demonstrate that the proposed approach reduces average beam steering overhead by 71% and decreases tracking overhead by 1.69%–11.15%, while maintaining high angular estimation accuracy.

Abstract

The detection of non-cooperative unmanned aerial vehicles (UAVs) presents significant challenges for Integrated Sensing and Communication (ISAC) systems due to the inherent limitations of single-modal perception and the competition for shared communication and sensing resources. To address these challenges, this paper proposes a novel Camera-Cooperative ISAC (CC-ISAC) framework that employs multimodal sensing to enable efficient UAV beam steering and tracking. The proposed framework employs cameras for coarse-grained airspace monitoring and utilizes ISAC for fine-grained, high-precision sensing, forming a complementary perception loop that enhances both sensing accuracy and resource efficiency. Within this framework, two key modules are developed: (1) a Vision-to-Echo Data Alignment (V2EDA) model that aligns visual and echo-domain features through cross-attention mechanisms, and (2) a Multimodal Fusion-Based Estimation (MMFE) model that integrates historical multimodal data with current observations for robust state estimation. Extensive evaluations conducted on the DeepSense 6G dataset demonstrate that the proposed framework achieves an average reduction of 71% in beam steering overhead and 1.69-11.15% in tracking overhead while maintaining high angular estimation accuracy. The CC-ISAC framework effectively mitigates resource contention between sensing and communication, enabling reliable UAV surveillance while freeing substantial system resources for additional communication tasks, thereby representing a practical advancement in ISAC system design.

Problem

• non-cooperative UAVs
• Integrated Sensing and Communication
• multimodal sensing
• resource contention
• beam steering

Innovation

• Camera-Cooperative ISAC
• Multimodal Sensing
• Vision-to-Echo Data Alignment
• Multimodal Fusion
• Non-Cooperative UAV Detection