SLIM-VDB: A Real-Time 3D Probabilistic Semantic Mapping Framework

2025-12-14

AI Summary

Existing 3D semantic mapping systems underutilize OpenVDB’s sparse voxel representation and struggle to jointly support closed-set category classification and open-vocabulary language labels (e.g., CLIP embeddings) within a unified inference framework. To address this, we propose the first OpenVDB-based real-time 3D semantic mapping framework leveraging Bayesian probabilistic updates. Our method integrates closed-set classification and open-vocabulary grounding via lightweight, online fusion of multi-source semantic evidence—encoded as sparse voxels—and employs a hybrid C++/Python architecture for efficiency. By exploiting spatial sparsity and performing incremental probabilistic integration, our approach significantly reduces memory footprint and fusion latency while preserving mapping accuracy. The framework is fully open-sourced and provides a scalable, unified solution for robot scene understanding—enabling seamless interoperability between traditional categorical semantics and foundation-model-derived language embeddings.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: 3D Computer Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

This paper introduces SLIM-VDB, a new lightweight semantic mapping system with probabilistic semantic fusion for closed-set or open-set dictionaries. Advances in data structures from the computer graphics community, such as OpenVDB, have demonstrated significantly improved computational and memory efficiency in volumetric scene representation. Although OpenVDB has been used for geometric mapping in robotics applications, semantic mapping for scene understanding with OpenVDB remains unexplored. In addition, existing semantic mapping systems lack support for integrating both fixed-category and open-language label predictions within a single framework. In this paper, we propose a novel 3D semantic mapping system that leverages the OpenVDB data structure and integrates a unified Bayesian update framework for both closed- and open-set semantic fusion. Our proposed framework, SLIM-VDB, achieves significant reduction in both memory and integration times compared to current state-of-the-art semantic mapping approaches, while maintaining comparable mapping accuracy. An open-source C++ codebase with a Python interface is available at https://github.com/umfieldrobotics/slim-vdb.

Problem

• Develops a lightweight 3D semantic mapping system for robotics
• Integrates closed-set and open-set semantic fusion in one framework
• Uses OpenVDB for efficient memory and computational performance

Innovation

• Uses OpenVDB for efficient 3D volumetric representation
• Integrates Bayesian update for closed and open-set semantics
• Achieves reduced memory and faster integration times