Gaussian-Det: Learning Closed-Surface Gaussians for 3D Object Detection

2024-10-02

arXiv.org

AI Summary

This paper addresses two key challenges in multi-view 3D object detection: coarse geometric representation and unreliable objectness reasoning. To this end, we propose a Gaussian distribution modeling framework based on closed surfaces—explicitly representing continuous, closed object surfaces via Gaussian rasterization, regularized by a surface-closure prior. Our core contributions are: (1) the first closed-surface Gaussian representation paradigm for 3D detection; and (2) an end-to-end differentiable Closure Inferring Module (CIM) that jointly estimates probabilistic residuals and aggregates global closure metrics to enable robust objectness inference and proposal refinement. By unifying multi-view geometry, probabilistic learning, and differentiable rendering, our method achieves state-of-the-art performance on both synthetic and real-world benchmarks—outperforming monocular and NeRF-based approaches in AP and Recall across all settings.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Skins wrapping around our bodies, leathers covering over the sofa, sheet metal coating the car - it suggests that objects are enclosed by a series of continuous surfaces, which provides us with informative geometry prior for objectness deduction. In this paper, we propose Gaussian-Det which leverages Gaussian Splatting as surface representation for multi-view based 3D object detection. Unlike existing monocular or NeRF-based methods which depict the objects via discrete positional data, Gaussian-Det models the objects in a continuous manner by formulating the input Gaussians as feature descriptors on a mass of partial surfaces. Furthermore, to address the numerous outliers inherently introduced by Gaussian splatting, we accordingly devise a Closure Inferring Module (CIM) for the comprehensive surface-based objectness deduction. CIM firstly estimates the probabilistic feature residuals for partial surfaces given the underdetermined nature of Gaussian Splatting, which are then coalesced into a holistic representation on the overall surface closure of the object proposal. In this way, the surface information Gaussian-Det exploits serves as the prior on the quality and reliability of objectness and the information basis of proposal refinement. Experiments on both synthetic and real-world datasets demonstrate that Gaussian-Det outperforms various existing approaches, in terms of both average precision and recall.

Problem

• Improving 3D object detection using continuous surface representation.
• Addressing outliers in Gaussian splatting with Closure Inferring Module.
• Enhancing objectness deduction via probabilistic feature residuals.

Innovation

• Gaussian Splatting for surfaces
• Closure Inferring Module
• Continuous 3D object modeling