Wave-Former: Through-Occlusion 3D Reconstruction via Wireless Shape Completion

2025-11-18

AI Summary

To address high-fidelity 3D shape reconstruction under complete occlusion, this paper proposes the first three-stage framework integrating millimeter-wave (mmWave) wireless sensing with vision-based shape completion. Methodologically, it pioneers embedding a physics-informed mmWave signal propagation model into a Transformer architecture and introduces an entropy-guided surface selection mechanism—enabling training solely on synthetic data while achieving strong generalization to real-world scenarios. Key technical innovations include: (i) occlusion-penetrating mmWave sensing, (ii) physics-driven signal modeling, and (iii) entropy-weighted surface optimization. On standard benchmarks, our method achieves a recall of 72% (+18 percentage points over prior work, from 54% to 72%) while maintaining 85% precision, substantially outperforming state-of-the-art approaches. This work establishes a new paradigm for concealed-object perception in robotic manipulation, augmented reality interaction, and intelligent logistics.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We present Wave-Former, a novel method capable of high-accuracy 3D shape reconstruction for completely occluded, diverse, everyday objects. This capability can open new applications spanning robotics, augmented reality, and logistics. Our approach leverages millimeter-wave (mmWave) wireless signals, which can penetrate common occlusions and reflect off hidden objects. In contrast to past mmWave reconstruction methods, which suffer from limited coverage and high noise, Wave-Former introduces a physics-aware shape completion model capable of inferring full 3D geometry. At the heart of Wave-Former's design is a novel three-stage pipeline which bridges raw wireless signals with recent advancements in vision-based shape completion by incorporating physical properties of mmWave signals. The pipeline proposes candidate geometric surfaces, employs a transformer-based shape completion model designed specifically for mmWave signals, and finally performs entropy-guided surface selection. This enables Wave-Former to be trained using entirely synthetic point-clouds, while demonstrating impressive generalization to real-world data.In head-to-head comparisons with state-of-the-art baselines, Wave-Former raises recall from 54% to 72% while maintaining a high precision of 85%.

Problem

• Reconstructing 3D shapes of completely occluded objects using wireless signals
• Overcoming limited coverage and high noise in millimeter-wave reconstruction methods
• Bridging wireless signals with vision-based shape completion for diverse objects

Innovation

• Uses millimeter-wave signals for occlusion penetration
• Introduces physics-aware transformer model for shape completion
• Implements three-stage pipeline with entropy-guided selection