RISE: Single Static Radar-based Indoor Scene Understanding

2025-11-17

AI Summary

To address the challenge of indoor scene understanding—specifically layout reconstruction and object detection—using single-static millimeter-wave (mmWave) radar, which suffers from inherently low spatial resolution, this paper introduces, for the first time, a dual-angle multipath enhancement framework that models multipath reflections as effective geometric signals. Our method jointly encodes angle-of-arrival (AoA) and angle-of-departure (AoD), and integrates a hierarchical diffusion generative model grounded in radar physics to enable robust simulation-to-real domain adaptation and structural inference. We further construct the first large-scale real-world dataset and end-to-end system framework dedicated to single-radar indoor understanding. Experiments demonstrate significant improvements: layout reconstruction Chamfer distance is reduced to 16 cm—a 60% reduction over baseline methods—and, for the first time, indoor object detection is achieved on mmWave radar with an IoU of 58%, substantially outperforming prior approaches.

Technology Category

• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Robust and privacy-preserving indoor scene understanding remains a fundamental open problem. While optical sensors such as RGB and LiDAR offer high spatial fidelity, they suffer from severe occlusions and introduce privacy risks in indoor environments. In contrast, millimeter-wave (mmWave) radar preserves privacy and penetrates obstacles, but its inherently low spatial resolution makes reliable geometric reasoning difficult. We introduce RISE, the first benchmark and system for single-static-radar indoor scene understanding, jointly targeting layout reconstruction and object detection. RISE is built upon the key insight that multipath reflections, traditionally treated as noise, encode rich geometric cues. To exploit this, we propose a Bi-Angular Multipath Enhancement that explicitly models Angle-of-Arrival and Angle-of-Departure to recover secondary (ghost) reflections and reveal invisible structures. On top of these enhanced observations, a simulation-to-reality Hierarchical Diffusion framework transforms fragmented radar responses into complete layout reconstruction and object detection. Our benchmark contains 50,000 frames collected across 100 real indoor trajectories, forming the first large-scale dataset dedicated to radar-based indoor scene understanding. Extensive experiments show that RISE reduces the Chamfer Distance by 60% (down to 16 cm) compared to the state of the art in layout reconstruction, and delivers the first mmWave-based object detection, achieving 58% IoU. These results establish RISE as a new foundation for geometry-aware and privacy-preserving indoor scene understanding using a single static radar.

Problem

• Achieving robust indoor scene understanding while preserving privacy using radar
• Overcoming low spatial resolution limitations of millimeter-wave radar systems
• Transforming multipath radar reflections into geometric layout and object detection

Innovation

• Uses multipath reflections for geometric cues
• Models angles to recover ghost reflections
• Applies hierarchical diffusion for layout reconstruction