MoCA3D: Monocular 3D Bounding Box Prediction in the Image Plane

2026-03-19

AI Summary

This work addresses the challenge of monocular 3D object detection in real-world scenarios where camera intrinsics are unknown, making accurate recovery of 3D geometry from the image plane difficult. To this end, we propose MoCA3D, a category-agnostic monocular 3D detection model that eliminates the need for camera intrinsics at inference time by directly predicting 3D bounding box corners and their depths densely in the image plane via corner heatmaps and depth maps. Departing from conventional paradigms that lift 2D RoIs to 3D boxes, our method introduces pixel-aligned geometric constraints, enabling high-fidelity, intrinsic-agnostic 3D geometry prediction for the first time. We further propose the PAG metric to evaluate consistency between predicted corners and depths, achieving a 22.8% improvement on this metric while matching state-of-the-art 3D IoU performance, with model parameters reduced by up to 57×, significantly enhancing applicability in settings lacking camera calibration.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Monocular 3D object understanding has largely been cast as a 2D RoI-to-3D box lifting problem. However, emerging downstream applications require image-plane geometry (e.g., projected 3D box corners) which cannot be easily obtained without known intrinsics, a problem for object detection in the wild. We introduce MoCA3D, a Monocular, Class-Agnostic 3D model that predicts projected 3D bounding box corners and per-corner depths without requiring camera intrinsics at inference time. MoCA3D formulates pixel-space localization and depth assignment as dense prediction via corner heatmaps and depth maps. To evaluate image-plane geometric fidelity, we propose Pixel-Aligned Geometry (PAG), which directly measures image-plane corner and depth consistency. Extensive experiments demonstrate that MoCA3D achieves state-of-the-art performance, improving image-plane corner PAG by 22.8% while remaining comparable on 3D IoU, using up to 57 times fewer trainable parameters. Finally, we apply MoCA3D to downstream tasks which were previously impractical under unknown intrinsics, highlighting its utility beyond standard baseline models.

Problem

• Monocular 3D object detection
• camera intrinsics
• image-plane geometry
• 3D bounding box
• projected corners

Innovation

• Monocular 3D detection
• Camera-intrinsic-free
• Dense prediction
• Pixel-Aligned Geometry
• Projected 3D corners