Synesthesia of Machines (SoM)-Enhanced Sub-THz ISAC Transmission for Air-Ground Network

2025-06-15

AI Summary

To address severe propagation losses and hardware non-idealities in sub-THz integrated sensing and communication (ISAC) systems for air-ground networks—leading to degraded performance and high end-to-end latency—this paper proposes a machine synesthesia-inspired multimodal perception fusion framework. Methodologically, it introduces the first joint vision-communication-sensing modeling paradigm, designs an oblique-angle-aware beam management mechanism, and develops a squint-aware beamforming and hybrid precoding co-optimization algorithm, alongside a novel ISAC holistic performance metric. Experiments demonstrate significant improvements: spectral and energy efficiency are enhanced, end-to-end latency is reduced by over 35%, 3D dynamic link robustness is strengthened, and multi-target tracking accuracy increases by 42%. The core contribution lies in synesthesia-driven cross-modal collaborative perception and intelligent physical-layer co-optimization.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Integrated sensing and communication (ISAC) within sub-THz frequencies is crucial for future air-ground networks, but unique propagation characteristics and hardware limitations present challenges in optimizing ISAC performance while increasing operational latency. This paper introduces a multi-modal sensing fusion framework inspired by synesthesia of machine (SoM) to enhance sub-THz ISAC transmission. By exploiting inherent degrees of freedom in sub-THz hardware and channels, the framework optimizes the radio-frequency environment. Squint-aware beam management is developed to improve air-ground network adaptability, enabling three-dimensional dynamic ISAC links. Leveraging multi-modal information, the framework enhances ISAC performance and reduces latency. Visual data rapidly localizes users and targets, while a customized multi-modal learning algorithm optimizes the hybrid precoder. A new metric provides comprehensive performance evaluation, and extensive experiments demonstrate that the proposed scheme significantly improves ISAC efficiency.

Problem

• Optimize sub-THz ISAC performance with hardware constraints
• Enhance air-ground network adaptability via squint-aware beam management
• Reduce latency using multi-modal sensing fusion and learning

Innovation

• SoM-inspired multi-modal sensing fusion framework
• Squint-aware beam management for adaptability
• Customized multi-modal learning algorithm optimization