DynOPETs: A Versatile Benchmark for Dynamic Object Pose Estimation and Tracking in Moving Camera Scenarios

2025-03-25

AI Summary

Existing benchmarks lack realistic datasets and precise 6D pose annotations for scenarios involving both dynamic objects and moving cameras. Method: We propose the first high-quality benchmark for 6D pose estimation and tracking in such collaborative settings, integrating multi-frame pose tracking, pose-graph optimization, and a pseudo-label self-generation–iterative refinement mechanism to achieve accurate, scalable 6D pose annotation. Contribution/Results: We introduce the first publicly available dataset supporting 6D pose evaluation under moving-camera conditions, comprising large-scale real-world sequences with precise annotations. Comprehensive evaluation across 18 state-of-the-art methods quantitatively reveals significant performance degradation under camera motion—a previously uncharacterized limitation. Our benchmark provides critical data infrastructure and standardized evaluation protocols to advance robust 6D pose estimation research.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Large-scale security measurements

Abstract

In the realm of object pose estimation, scenarios involving both dynamic objects and moving cameras are prevalent. However, the scarcity of corresponding real-world datasets significantly hinders the development and evaluation of robust pose estimation models. This is largely attributed to the inherent challenges in accurately annotating object poses in dynamic scenes captured by moving cameras. To bridge this gap, this paper presents a novel dataset DynOPETs and a dedicated data acquisition and annotation pipeline tailored for object pose estimation and tracking in such unconstrained environments. Our efficient annotation method innovatively integrates pose estimation and pose tracking techniques to generate pseudo-labels, which are subsequently refined through pose graph optimization. The resulting dataset offers accurate pose annotations for dynamic objects observed from moving cameras. To validate the effectiveness and value of our dataset, we perform comprehensive evaluations using 18 state-of-the-art methods, demonstrating its potential to accelerate research in this challenging domain. The dataset will be made publicly available to facilitate further exploration and advancement in the field.

Problem

• Lack of real-world datasets for dynamic object pose estimation with moving cameras
• Challenges in accurately annotating object poses in dynamic moving-camera scenes
• Need for robust benchmark to evaluate pose estimation and tracking methods

Innovation

• Novel dataset DynOPETs for dynamic scenes
• Pseudo-label generation via pose estimation and tracking
• Pose graph optimization for annotation refinement