LOST-3DSG: Lightweight Open-Vocabulary 3D Scene Graphs with Semantic Tracking in Dynamic Environments

2026-01-06

arXiv.org

AI Summary

This work addresses the high computational cost of tracking moving objects in dynamic environments by proposing a lightweight, open-vocabulary 3D scene graph method. It introduces word2vec and sentence embeddings into 3D scene understanding for the first time, replacing high-dimensional CLIP visual features with low-dimensional semantic representations to enable efficient, continuous object tracking. By avoiding the storage of expensive visual embeddings, the approach significantly reduces both computational and memory overhead while preserving strong generalization to unseen categories. Experimental results on a real TIAGo robotic platform demonstrate that the proposed method achieves superior tracking efficiency and performance compared to existing approaches relying on high-dimensional visual embeddings.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Tracking objects that move within dynamic environments is a core challenge in robotics. Recent research has advanced this topic significantly; however, many existing approaches remain inefficient due to their reliance on heavy foundation models. To address this limitation, we propose LOST-3DSG, a lightweight open-vocabulary 3D scene graph designed to track dynamic objects in real-world environments. Our method adopts a semantic approach to entity tracking based on word2vec and sentence embeddings, enabling an open-vocabulary representation while avoiding the necessity of storing dense CLIP visual features. As a result, LOST-3DSG achieves superior performance compared to approaches that rely on high-dimensional visual embeddings. We evaluate our method through qualitative and quantitative experiments conducted in a real 3D environment using a TIAGo robot. The results demonstrate the effectiveness and efficiency of LOST-3DSG in dynamic object tracking. Code and supplementary material are publicly available on the project website at https://lab-rococo-sapienza.github.io/lost-3dsg/.

Problem

• dynamic object tracking
• 3D scene graph
• open-vocabulary
• lightweight representation
• semantic tracking

Innovation

• lightweight
• open-vocabulary
• 3D scene graph
• semantic tracking
• dynamic object tracking