TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers

2026-08-15

AI Summary

This study addresses the challenge of real-time 6DoF pose estimation on resource-constrained hardware by proposing a lightweight, end-to-end, single-stage framework based on LW-DETR. This approach introduces a post-processing-free Transformer architecture that jointly performs detection and pose regression via set prediction, specialized MLP heads, and a symmetry-aware Hungarian matcher, thereby eliminating non-differentiable modules such as PnP and NMS. Experimental results demonstrate that the model achieves an ADD-S AUC of 85.9% on the YCB-V dataset while reducing parameter count by 72.7%. Notably, it attains an inference latency of only 4.5 ms on a Jetson Nano, enabling high-precision, real-time deployment on edge devices.

Abstract

Real-time 6DoF object pose estimation on resource-constrained hardware remains challenging, as accurate correspondence-based and refinement pipelines typically rely on non-differentiable PnP/RANSAC stages or costly iterative refinement, while recent foundation-model-based approaches incur inference costs that are prohibitive for edge deployment. We present TinyDETR-Pose, a lightweight, end-to-end, single-stage framework that jointly detects objects and regresses their full 6D pose in a single forward pass. Built on the efficient LW-DETR architecture, TinyDETR-Pose formulates detection and pose estimation as a set-prediction problem and attaches dedicated MLP heads for rotation, monocular depth, and projected object center regression to each decoder query, eliminating the need for PnP, NMS (non-maximum suppression), or iterative pose refinement. Object symmetries are handled through a ADD-S loss applied uniformly to all objects, without the need for object-specific loss schedules or separate geodesic/ADD supervision. In addition, predictions are assigned to ground truth using a symmetry-safe Hungarian matcher based on class and 2D spatial cues, yielding stable assignment under symmetry and depth ambiguity. On YCB-V, TinyDETR-Pose achieves a comparable ADD-S AUC of 85.9, while requiring up to 72.7% fewer parameters than other DETR-based single-stage pose-estimation approaches. Due to its compact design, TinyDETR-Pose runs in real time and achieves an inference latency of only ~4.5 ms per frame on an NVIDIA Jetson Nano using TensorRT, demonstrating that accurate end-to-end transformer-based 6D pose estimation can be made practical for edge deployment.

Problem

• 6DoF object pose estimation
• real-time inference
• resource-constrained hardware
• edge deployment

Innovation

• End-to-End 6DoF Pose Estimation
• Lightweight Transformer
• Symmetry-Safe Hungarian Matcher
• Single-Stage Set Prediction
• Edge Deployment