RadarGen: Automotive Radar Point Cloud Generation from Cameras

2025-12-19

AI Summary

This work addresses the challenging cross-modal generation problem of synthesizing high-fidelity automotive radar point clouds from multi-view camera images. To this end, we propose the first diffusion-based framework tailored for radar point cloud generation. Our method innovatively adapts image latent-space diffusion models to the radar domain, introducing a two-stage paradigm: (1) bird’s-eye-view (BEV) radar map generation, followed by (2) lightweight point cloud reconstruction. We jointly incorporate depth, semantic, and optical flow priors, while embedding radar-specific physical modeling—including radar cross-section (RCS) and Doppler effects—to enforce structural, reflective, and kinematic consistency. Evaluated on large-scale real-world vehicle data, our generated point clouds faithfully reproduce the statistical distribution of real radar measurements. Crucially, perception models trained on synthetic radar data achieve significantly improved performance—closing the gap with those trained on real radar data by an average +12.3% mAP. This establishes a novel, low-cost, and scalable paradigm for multimodal autonomous driving simulation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery. RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form that encodes spatial structure together with radar cross section (RCS) and Doppler attributes. A lightweight recovery step reconstructs point clouds from the generated maps. To better align generation with the visual scene, RadarGen incorporates BEV-aligned depth, semantic, and motion cues extracted from pretrained foundation models, which guide the stochastic generation process toward physically plausible radar patterns. Conditioning on images makes the approach broadly compatible, in principle, with existing visual datasets and simulation frameworks, offering a scalable direction for multimodal generative simulation. Evaluations on large-scale driving data show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data, marking a step toward unified generative simulation across sensing modalities.

Problem

• Generates radar point clouds from camera images
• Aligns radar synthesis with visual scene cues
• Reduces gap to perception models using real data

Innovation

• Diffusion model synthesizes radar point clouds from camera images
• Bird's-eye-view representation encodes radar attributes and spatial structure
• BEV-aligned depth and semantic cues guide physically plausible generation