Problem
Research questions and friction points this paper is trying to address.
video segmentation
object dynamicity
camera ego-motion
region-level prediction
query-based
DynEoMT: Learning Object Dynamicity from Online Segmentation Queries
🤖 AI Summary
本文提出DynEoMT方法，通过在线查询增强视频分割模型以预测区域动态性，解决了无法仅从语义推断物体是否独立于相机移动的问题。
📝 Abstract
Video segmentation models recognize and track objects over time, but they do not indicate whether each segmented region moves independently of the observing camera. This dynamicity attribute cannot be inferred from semantics alone and is confounded by camera ego-motion. We introduce \method, an online framework that augments query-based video segmentation with region-level dynamicity prediction. It jointly produces the original segmentation outputs and a dynamic or static state for each predicted region. At inference, DynEoMT uses only the current frame and propagated queries, without optical flow, depth, camera pose, previous RGB frames, or feature maps. Because established video segmentation benchmarks do not annotate this attribute, we also introduce a class-agnostic offline supervision pipeline using camera-compensated optical flow and confidence-aware temporal filtering. Across VIPSeg, OVIS, YouTube-VIS 2022, and VSPW, DynEoMT achieves balanced accuracies of 84.3, 68.0, 68.6, and 87.6, respectively, while largely preserving segmentation performance. These results show that segmentation-region dynamicity can be learned from propagated queries, enabling its online prediction without a dedicated motion-processing pipeline at inference. The complete code will be released as open source to enable full reproduction of the method and experiments.
Innovation
Methods, ideas, or system contributions that make the work stand out.
online framework
region-level dynamicity prediction
query-based video segmentation
camera-compensated optical flow
confidence-aware temporal filtering
🔎 Similar Papers
No similar papers found.
