MM-2FSK: Multimodal Frequency Shift Keying for Ultra-Efficient and Robust High-Resolution MIMO Radar Imaging

2025-11-03

AI Summary

To address the challenges of bandwidth limitation, high computational complexity, and low frame rate in high-resolution MIMO radar imaging of dynamic scenes—particularly for joint reconstruction of static and fast-moving targets—this paper proposes a multimodal dual-frequency-shift-keying (MM-2FSK) method integrating millimeter-wave MIMO radar with optical depth sensing. For the first time, optical depth priors are leveraged to guide sparse frequency sampling and reconstruction, eliminating reliance on conventional prior knowledge of target positions and depth ranges. Through synergistic optimization of spatial registration and sparse signal recovery, the method achieves high-fidelity 3D imaging while significantly reducing RF bandwidth and sampling rate. Experiments on real-world data demonstrate reconstruction accuracy comparable to full-bandwidth sampling, a 3.2× improvement in frame rate, and a 68% reduction in computational time—enabling high-precision, real-time 3D reconstruction of both static and dynamic targets.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Accurate reconstruction of static and rapidly moving targets demands three-dimensional imaging solutions with high temporal and spatial resolution. Radar sensors are a promising sensing modality because of their fast capture rates and their independence from lighting conditions. To achieve high spatial resolution, MIMO radars with large apertures are required. Yet, they are infrequently used for dynamic scenarios due to significant limitations in signal processing algorithms. These limitations impose substantial hardware constraints due to their computational intensity and reliance on large signal bandwidths, ultimately restricting the sensor's capture rate. One solution of previous work is to use few frequencies only, which enables faster capture and requires less computation; however, this requires coarse knowledge of the target's position and works in a limited depth range only. To address these challenges, we extend previous work into the multimodal domain with MM-2FSK, which leverages an assistive optical depth sensing modality to obtain a depth prior, enabling high framerate capture with only few frequencies. We evaluate our method using various target objects with known ground truth geometry that is spatially registered to real millimeter-wave MIMO radar measurements. Our method demonstrates superior performance in terms of depth quality, being able to compete with the time- and resource-intensive measurements with many frequencies.

Problem

• Achieving high-resolution 3D radar imaging with limited frequencies
• Overcoming hardware constraints from computationally intensive signal processing
• Enabling robust imaging of dynamic targets without large bandwidth

Innovation

• Multimodal 2FSK combines radar with optical depth sensing
• Uses few frequencies for high framerate MIMO radar imaging
• Leverages depth prior to enable robust high-resolution reconstruction