m^3TrackFormer: Transformer-based mmWave Multi-Target Tracking with Lost Target Re-Acquisition Capability

2026-02-20

AI Summary

In millimeter-wave integrated sensing and communication (ISAC) systems, narrow beams are prone to losing track of multiple targets due to position mismatch, particularly under complex trajectories where conventional Kalman filtering fails. To address this challenge, this work proposes a Transformer-based dual-mode adaptive tracking framework comprising a Normal-tracking Network (N-Net) and a Reacquisition Network (R-Net), built upon Transformer encoders and decoders, respectively. The framework integrates beam prediction with an adaptive scanning strategy to effectively capture global trajectory features. It enables rapid reacquisition of lost targets with low scanning overhead while maintaining stable tracking of other targets, thereby significantly enhancing beam prediction accuracy.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This paper considers a millimeter wave (mmWave) integrated sensing and communication (ISAC) system, where a base station (BS) equipped with a large number of antennas but a small number of radio-frequency (RF) chains emits pencillike narrow beams for persistent tracking of multiple moving targets. Under this model, the tracking lost issue arising from the misalignment between the pencil-like beams and the true target positions is inevitable, especially when the trajectories of the targets are complex, and the conventional Kalman filter-based scheme does not work well. To deal with this issue, we propose a Transformer-based mmWave multi-target tracking framework, namely m3TrackFormer, with a novel re-acquisition mechanism, such that even if the echo signals from some targets are too weak to extract sensing information, we are able to re-acquire their locations quickly with small beam sweeping overhead. Specifically, the proposed framework can operate in two modes of normal tracking and target re-acquisition during the tracking procedure, depending on whether the tracking lost occurs. When all targets are hit by the swept beams, the framework works in the Normal Tracking Mode (N-Mode) with a Transformer encoder-based Normal Tracking Network (N-Net) to accurately estimate the positions of these targets and predict the swept beams in the next time block. While the tracking lost happens, the framework will switch to the Re-Acquisition Mode (R-Mode) with a Transformer decoder-based Re-Acquisition Network (RNet) to adjust the beam sweeping strategy for getting back the lost targets and maintaining the tracking of the remaining targets. Thanks to the ability of global trajectory feature extraction, the m3TrackFormer can achieve high beam prediction accuracy and quickly re-acquire the lost targets, compared with other tracking methods.

Problem

• mmWave
• multi-target tracking
• tracking lost
• beam misalignment
• ISAC

Innovation

• Transformer-based tracking
• mmWave ISAC
• lost target re-acquisition
• beam prediction
• multi-target tracking