CaRLi-V: Camera-RADAR-LiDAR Point-Wise 3D Velocity Estimation

2025-11-03

AI Summary

This work addresses the challenge of dense, point-wise 3D velocity estimation for robots interacting with non-rigid dynamic objects (e.g., pedestrians) in complex environments. We propose a tightly coupled RADAR–LiDAR–camera fusion framework. Its core innovation is the first-ever construction of a RADAR Velocity Cube (RVC), enabling dense modeling of radial velocity directly from raw RADAR measurements. Leveraging 2D motion priors from optical flow, high-accuracy depth from LiDAR, and multi-sensor geometric constraints, we formulate and solve for per-point 3D velocity in closed form. Implemented as an open-source ROS2 system, our method is evaluated on a custom multimodal dataset: it achieves a mean velocity error below 0.15 m/s, operates in real time, and delivers high-accuracy, dense 3D velocity fields over point clouds—significantly enhancing robustness in dynamic obstacle avoidance and trajectory planning.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Accurate point-wise velocity estimation in 3D is crucial for robot interaction with non-rigid, dynamic agents, such as humans, enabling robust performance in path planning, collision avoidance, and object manipulation in dynamic environments. To this end, this paper proposes a novel RADAR, LiDAR, and camera fusion pipeline for point-wise 3D velocity estimation named CaRLi-V. This pipeline leverages raw RADAR measurements to create a novel RADAR representation, the velocity cube, which densely represents radial velocities within the RADAR's field-of-view. By combining the velocity cube for radial velocity extraction, optical flow for tangential velocity estimation, and LiDAR for point-wise range measurements through a closed-form solution, our approach can produce 3D velocity estimates for a dense array of points. Developed as an open-source ROS2 package, CaRLi-V has been field-tested against a custom dataset and proven to produce low velocity error metrics relative to ground truth, enabling point-wise velocity estimation for robotic applications.

Problem

• Estimating 3D point-wise velocity for dynamic agents
• Fusing camera, RADAR, and LiDAR for velocity estimation
• Enabling robotic path planning in dynamic environments

Innovation

• Fuses camera RADAR LiDAR for 3D velocity
• Uses novel RADAR velocity cube representation
• Combines velocity cube optical flow LiDAR