Problem
Research questions and friction points this paper is trying to address.
Referring Multi-Object Tracking
Multimodal Large Language Models
Yes/No Verification
YesTrack: Referring Multi-Object Tracking via MLLM-based Yes/No Verification
🤖 AI Summary
针对现有方法未充分利用多模态大语言模型(MLLM)的问题，提出YesTrack方法，通过MLLM进行是/否验证，并引入时间一致性约束提高跟踪效率和准确性。
📝 Abstract
Referring multi-object tracking (RMOT) aims to track every instance in a video that matches a given language expression. Despite the recent integration of multimodal large language models (MLLMs) to enhance generalization, existing methods predominantly relegate them to the role of caption generators, necessitating external modules for final decision-making. This paradigm not only introduces extra latency but also severely underutilizes the inherent vision-language alignment capabilities of MLLMs. To address these limitations, we propose YesTrack, a novel two-stage RMOT method that reformulates referring as a discriminative task, directly leveraging MLLMs for Yes/No verification without explicit text generation. To further enhance the reliability and efficiency of this MLLM-based verification, we introduce two lightweight temporal consistency constraints: Temporal Confidence Prior (TCP) and Temporal Reference Propagation (TRP). We further validate the generality of this discriminative paradigm by proposing YesTrack-MOT, a straightforward yet highly effective instantiation for generic multi-object tracking (MOT). Experiments on Refer-KITTI and Refer-KITTI-V2 show that YesTrack significantly outperforms existing state-of-the-art methods while maintaining high efficiency, even when implemented with the smallest variant of Qwen3-VL. Code is released at https://github.com/ggbondrighthere24/YesTrack.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Yes/No Verification
Temporal Consistency Constraints
Multimodal Large Language Models
Referring Multi-Object Tracking
🔎 Similar Papers
No similar papers found.
