Mono3DV: Monocular 3D Object Detection with 3D-Aware Bipartite Matching and Variational Query DeNoising

2026-01-03

arXiv.org

AI Summary

This work addresses a critical limitation in existing DETR-based monocular 3D object detection methods, which rely solely on 2D criteria during bipartite matching and consequently suppress high-quality 3D predictions by neglecting 3D geometric properties. To overcome this, the authors propose Mono3DV, a novel framework that introduces, for the first time, a 3D-aware bipartite matching mechanism that explicitly incorporates 3D geometric information into the matching cost. Additionally, they design a variational query denoising strategy to stabilize training and mitigate gradient vanishing. The resulting end-to-end trainable model achieves state-of-the-art performance on the KITTI 3D object detection benchmark without leveraging any external data.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

While DETR-like architectures have demonstrated significant potential for monocular 3D object detection, they are often hindered by a critical limitation: the exclusion of 3D attributes from the bipartite matching process. This exclusion arises from the inherent ill-posed nature of 3D estimation from monocular image, which introduces instability during training. Consequently, high-quality 3D predictions can be erroneously suppressed by 2D-only matching criteria, leading to suboptimal results. To address this, we propose Mono3DV, a novel Transformer-based framework. Our approach introduces three key innovations. First, we develop a 3D-Aware Bipartite Matching strategy that directly incorporates 3D geometric information into the matching cost, resolving the misalignment caused by purely 2D criteria. Second, it is important to stabilize the Bipartite Matching to resolve the instability occurring when integrating 3D attributes. Therefore, we propose 3D-DeNoising scheme in the training phase. Finally, recognizing the gradient vanishing issue associated with conventional denoising techniques, we propose a novel Variational Query DeNoising mechanism to overcome this limitation, which significantly enhances model performance. Without leveraging any external data, our method achieves state-of-the-art results on the KITTI 3D object detection benchmark.

Problem

• monocular 3D object detection
• bipartite matching
• 3D estimation instability
• DETR-like architectures

Innovation

• 3D-Aware Bipartite Matching
• Variational Query DeNoising
• Monocular 3D Object Detection
• Transformer-based Framework
• 3D DeNoising