Hydra++: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation

2026-07-10

AI Summary

Existing 3D scene graph systems struggle to capture instance-level geometric details due to their reliance on local point clouds or category-level CAD templates. This work proposes a novel approach that integrates learning-driven object shape estimation with reprojection mask consistency verification within a hierarchical 3D scene graph construction pipeline, supporting online, multimodal (LiDAR-camera) inputs. Building upon the CRISP framework, the system modularly incorporates category-agnostic shape estimators such as SAM3D and employs a consistency-based mechanism to filter out degenerate predictions. Evaluated in both simulated and real-world outdoor campus environments, the method substantially improves reconstruction quality at both the object and scene levels.

Abstract

3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities, such as objects and places, and their relationships. However, existing scene graph systems model object geometry coarsely, relying on partial point clouds or class-level CAD templates, which limits instance-specific shape detail. This paper presents Hydra++, a system-level investigation into how learning-based object shape estimators can be integrated into a hierarchical 3D scene graph pipeline. Hydra++ incorporates category-agnostic shape estimation and a reprojection-mask consistency check to reject degenerate predictions from partial observations or imprecise segmentation. In its default CRISP-based configuration, Hydra++ performs online scene graph construction; slower estimators such as SAM3D are evaluated as modular alternatives to demonstrate generalization-latency trade-offs. Furthermore, to address the challenges of sparse and noisy depth measurements in outdoor environments, Hydra++ supports a hybrid LiDAR-camera configuration for large-scale operation, improving scene-level reconstruction quality. Experiments in both simulation and real-world outdoor campus scenarios demonstrate that Hydra++ improves object- and scene-level reconstruction quality. Project page is available at https://hydra-plusplus.github.io/.

Problem

• 3D scene graph
• object-level shape estimation
• instance-specific geometry
• outdoor reconstruction
• sparse depth

Innovation

• category-agnostic shape estimation
• reprojection-mask consistency
• hierarchical 3D scene graph
• hybrid LiDAR-camera
• real-time scene reconstruction