Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking

2026-09-08

AI Summary

为解决在复杂环境下移动物体的感知与跟踪问题，本文提出了一种结合RGB、热成像和雷达数据的方法SAM-Radar，通过直接利用雷达测量的速度信息来提高目标分割和跟踪的准确性。

Abstract

Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motion from appearance, optical flow, or estimated trajectories lose that evidence under poor illumination, adverse weather, reflections, and occlusion. Radar is a natural remedy because it measures radial velocity directly instead of inferring it from photometric correspondence. However, existing benchmarks do not jointly provide radar measurements, dense moving-instance masks, and temporally consistent identities for surveillance. We therefore introduce RGBTR-Motion, a synchronized and calibrated fixed-camera benchmark that pairs RGB, thermal, and radar streams with dense instance masks and temporally consistent identities across diverse surveillance scenes. We also develop SAM-Radar, an RGB, thermal, and radar-based segmentation and tracking framework built on SAM 3. SAM-Radar's radar-aware detector fuses calibrated RGBT features with radar returns that are grounded at their projected image locations, and motion supervision, implemented as foreground classification of those projected returns, teaches the detector to reject clutter without any text prompt. The tracker associates accepted radar returns with individual trajectories and uses them as physical evidence that a visually degraded target remains present. This allows it to bridge short periods of low visibility or occlusion and reconnect a reappearing target to its existing identity instead of starting a new track. SAM-Radar attains 0.7027 IoU and 0.8090 F1-50, and raises MOTA, HOTA, and IDF1 by 0.2977, 0.1603, and 0.2857 over the strongest competing values.

Problem

• Moving-object perception
• Radar
• Temporal consistency
• Adverse conditions
• Benchmarks

Innovation

• Radar-based Segmentation
• Multimodal Fusion
• Temporal Consistency
• Motion Supervision