Cross-Modal Generation: From Commodity WiFi to High-Fidelity mmWave and RFID Sensing

2026-04-17

AI Summary

This work addresses the challenge of scarce mmWave and RFID signal data, which hinders the training of high-quality generative models. To overcome this limitation, the authors propose RF-CMG, a diffusion-based framework that leverages readily available WiFi signals to cross-modally synthesize high-fidelity mmWave and RFID signals. The key innovation lies in decoupling cross-modal generation into high-frequency distribution learning and low-frequency structural constraints, implemented through a Modality-Guided Embedding (MGE) module and a Low-Frequency Modality Consistency (LFMC) module. These components effectively mitigate source modality bias while preserving physical structure consistency. Experimental results demonstrate that the generated signals surpass those of existing generative models in quality and significantly enhance performance on downstream gesture recognition tasks.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

AIGC has shown remarkable success in CV and NLP, and has recently demonstrated promising potential in the wireless domain. However, significant data imbalance exists across RF modalities, with abundant WiFi data but scarce mmWave and RFID data due to high acquisition cost. This makes it difficult to train high-quality generative models for these data-scarce modalities. In this work, we propose RF-CMG, a diffusion-based cross-modal generative method that leverages data-rich WiFi signals to synthesize high-fidelity RF data for scarce modalities including mmWave and RFID. The key insight of RF-CMG is to decouple cross-modal generation into high-frequency guidance and low-frequency constraint, which respectively learn high-frequency distribution from limited target modality data and preserve the underlying physical structure via low-frequency constraints during generation. On this basis, we introduce a Modality-Guided Embedding (MGE) module to steer the reverse diffusion trajectory toward the target high-frequency distribution, and a Low-Frequency Modality Consistency (LFMC) module to progressively enforce low-frequency constraints to suppress the accumulation of source-modality structural biases during inference, enabling high-quality target-modality generation. Performance comparison with several prevalent generative models demonstrates that RF-CMG achieves superior performance in synthesizing RFID and mmWave signals. We further showcase the effectiveness of the data generated by RF-CMG in gesture recognition tasks, and analyze the impact of the proportion of synthetic data on downstream performance.

Problem

• cross-modal generation
• data scarcity
• mmWave
• RFID
• WiFi

Innovation

• cross-modal generation
• diffusion model
• RF sensing
• modality-guided embedding
• low-frequency consistency