Knowledge Graph Completion for Action Prediction on Situational Graphs -- A Case Study on Household Tasks

2025-08-19

AI Summary

Action link prediction in home activity situation knowledge graphs—often derived from video parsing—is severely hindered by pervasive information incompleteness, rendering conventional static link prediction methods ineffective (even underperforming random baselines) due to the graphs’ inherent sparsity, dynamic evolution, and strong temporal dependencies. Method: We propose the first situation-aware knowledge graph completion framework tailored for domestic tasks, integrating action semantic constraints with multi-granularity temporal modeling, and instantiate it via a lightweight situation-aware graph neural network. Contribution/Results: Evaluated on a benchmark dataset constructed from real-world home videos, our model achieves a +12.3% improvement in Mean Reciprocal Rank (MRR) over state-of-the-art methods. Ablation studies confirm its efficacy in enhancing robot task planning and video situation understanding. This work exposes the fundamental limitations of static link prediction paradigms for behavioral modeling and establishes a new paradigm for dynamic situational knowledge reasoning.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Knowledge Graphs are used for various purposes, including business applications, biomedical analyses, or digital twins in industry 4.0. In this paper, we investigate knowledge graphs describing household actions, which are beneficial for controlling household robots and analyzing video footage. In the latter case, the information extracted from videos is notoriously incomplete, and completing the knowledge graph for enhancing the situational picture is essential. In this paper, we show that, while a standard link prediction problem, situational knowledge graphs have special characteristics that render many link prediction algorithms not fit for the job, and unable to outperform even simple baselines.

Problem

• Completing situational knowledge graphs for household action prediction
• Addressing incompleteness in video-extracted information for robotics
• Overcoming limitations of standard link prediction algorithms

Innovation

• Knowledge graph completion for action prediction
• Addressing incompleteness in situational graphs
• Tailoring link prediction for household tasks