Long-Term Planning Around Humans in Domestic Environments with 3D Scene Graphs

2025-03-12

AI Summary

In home environments, dynamic human activities continually alter object states and spatial constraints, posing dual challenges for long-horizon robot navigation: maintaining task efficiency while ensuring social acceptability. To address this, we propose the Activity-Driven 3D Scene Graph (AD-3DSG), the first framework to explicitly model human activity preferences and behavioral semantics as quantifiable spatial influence factors. AD-3DSG integrates vision-language models for context-aware scene understanding and employs activity-relation reasoning coupled with semantic-cost-guided trajectory optimization to dynamically adjust navigation costs in human-activity-affected regions. Experiments demonstrate that our method significantly outperforms baselines in task completion rate, spatial adaptability, and social compliance—achieving more contextually grounded, human-aware navigation in domestic settings.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Search and Optimization: Adversarial Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Long-term planning for robots operating in domestic environments poses unique challenges due to the interactions between humans, objects, and spaces. Recent advancements in trajectory planning have leveraged vision-language models (VLMs) to extract contextual information for robots operating in real-world environments. While these methods achieve satisfying performance, they do not explicitly model human activities. Such activities influence surrounding objects and reshape spatial constraints. This paper presents a novel approach to trajectory planning that integrates human preferences, activities, and spatial context through an enriched 3D scene graph (3DSG) representation. By incorporating activity-based relationships, our method captures the spatial impact of human actions, leading to more context-sensitive trajectory adaptation. Preliminary results demonstrate that our approach effectively assigns costs to spaces influenced by human activities, ensuring that the robot trajectory remains contextually appropriate and sensitive to the ongoing environment. This balance between task efficiency and social appropriateness enhances context-aware human-robot interactions in domestic settings. Future work includes implementing a full planning pipeline and conducting user studies to evaluate trajectory acceptability.

Problem

• Long-term robot planning in human-inhabited domestic environments.
• Incorporating human activities and preferences in trajectory planning.
• Enhancing context-aware human-robot interactions using 3D scene graphs.

Innovation

• Integrates human preferences via 3D scene graphs
• Captures spatial impact of human activities
• Enhances context-aware human-robot interactions