Temporally Consistent Object 6D Pose Estimation for Robot Control

2026-05-04

AI Summary

This work addresses the challenge of achieving temporally consistent and robust 6D object pose estimation from monocular RGB images, a critical requirement for stable visual feedback control in robotics. The authors propose a factor graph–based online optimization framework that, for the first time, jointly models object motion dynamics and pose measurement uncertainty. By integrating outlier rejection with an online smoothing strategy, the method delivers temporally coherent and robust pose tracking. Evaluated on standard benchmarks, the approach significantly improves pose estimation accuracy and demonstrates enhanced system stability in vision-based force-controlled robotic manipulation tasks, thereby providing reliable state estimates for robot control.

Abstract

Single-view RGB object pose estimators have reached a level of precision and efficiency that makes them good candidates for vision-based robot control. However, off-the-shelf methods lack temporal consistency and robustness that are mandatory for a stable feedback control. In this work, we develop a factor graph approach to enforce temporal consistency of the object pose estimates. In particular, the proposed approach: (i) incorporates object motion models, (ii) explicitly estimates the object pose measurement uncertainty, and (iii) integrates the above two components in an online optimization-based estimator. We demonstrate that with appropriate outlier rejection and smoothing using the proposed factor graph approach, we can significantly improve the results on standardized pose estimation benchmarks. We experimentally validate the stability of the proposed approach for a feedback-based robot control task in which the object is tracked by the camera attached to a torque controlled manipulator.

Problem

• temporal consistency
• 6D pose estimation
• robot control
• robustness
• feedback control

Innovation

• temporal consistency
• 6D pose estimation
• factor graph
• motion model
• uncertainty estimation