NoPA: Non-Parametric Online 3D Scene Graph Generation

2026-07-01

AI Summary

Existing methods for 3D scene graph generation struggle to balance real-time performance and geometric fidelity due to their reliance on point cloud reconstruction or Gaussian approximations. This work proposes a non-parametric online generation approach that represents objects using a fixed set of particles, enabling efficient inference while preserving fine geometric details. The method innovatively employs kernel density estimation–based Maximum Mean Discrepancy (MMD) for low-complexity, robust object fusion and introduces a high-affinity–driven relation propagation mechanism to effectively mitigate relation loss caused by misclassification. By integrating these components, the proposed framework achieves significant improvements in both accuracy and geometric completeness of generated 3D scene graphs without compromising real-time performance.

Abstract

Classic 3D scene graph generation approaches fail to work in real-time due to the heavy computational cost of environment mapping and the need to generate intermediate point-cloud representations. To alleviate this issue, a recent work eschews point clouds in favor of a lightweight Gaussian distribution for each object. This approximation drastically speeds up inference and enables real-time 3D scene graph generation. However, the representation has two key weaknesses. \textbf{1)} Each object is approximated by a single 3D Gaussian, which causes a severe loss of 3D geometric detail. \textbf{2)} The discrepancy between this approximation and the true object geometry exacerbates the inaccurate merging of object candidates during online inference. To address these issues, we propose \textbf{NoPA}, which represents each object as a separate non-parametric distribution. This formulation retains 3D geometric information while preserving real-time inference of the parametric Gaussian formulation. To build upon our novel object representation, we propose a tailored merging strategy to recover coherent object instances. Specifically, we leverage maximum mean discrepancy on kernel density estimates to enable robust merging of object candidates during online exploration while minimizing added computational complexity. The key is to maintain a fixed particle set per object. Furthermore, to rectify the relation loss caused by misclassified objects, NoPA propagates relationships between objects with high affinity. Experiments show that NoPA substantially outperforms current methods without sacrificing real-time inference speed.

Problem

• 3D scene graph generation
• real-time inference
• geometric detail loss
• object merging
• non-parametric representation

Innovation

• non-parametric representation
• real-time 3D scene graph
• object merging
• maximum mean discrepancy
• kernel density estimation