Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback

2025-01-13

AI Summary

This work addresses the challenges of robot explainability and safety in human-robot collaborative scenarios. We propose a two-stage task specification learning framework: first, action preconditions, constraints, and effects are automatically induced from a small set of non-repetitive video-kinematic demonstrations using Inductive Logic Programming (ILP) and the Event Calculus; second, lightweight commonsense knowledge modeling is integrated with an online user-feedback-driven knowledge refinement mechanism to enable interpretable and formally verifiable rule evolution. The method unifies unsupervised action recognition with multimodal parsing and is validated in safety-critical domains such as surgical robotics. Compared to baselines, our approach significantly improves task specification accuracy while exhibiting low data dependency, high robustness, and cross-domain generalizability. To our knowledge, this is the first systematic integration of ILP with closed-loop human feedback for real-world robotic task induction.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Safety and Robustness
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

The increasing level of autonomy of robots poses challenges of trust and social acceptance, especially in human-robot interaction scenarios. This requires an interpretable implementation of robotic cognitive capabilities, possibly based on formal methods as logics for the definition of task specifications. However, prior knowledge is often unavailable in complex realistic scenarios. In this paper, we propose an offline algorithm based on inductive logic programming from noisy examples to extract task specifications (i.e., action preconditions, constraints and effects) directly from raw data of few heterogeneous (i.e., not repetitive) robotic executions. Our algorithm leverages on the output of any unsupervised action identification algorithm from video-kinematic recordings. Combining it with the definition of very basic, almost task-agnostic, commonsense concepts about the environment, which contribute to the interpretability of our methodology, we are able to learn logical axioms encoding preconditions of actions, as well as their effects in the event calculus paradigm. Since the quality of learned specifications depends mainly on the accuracy of the action identification algorithm, we also propose an online framework for incremental refinement of task knowledge from user feedback, guaranteeing safe execution. Results in a standard manipulation task and benchmark for user training in the safety-critical surgical robotic scenario, show the robustness, data- and time-efficiency of our methodology, with promising results towards the scalability in more complex domains.

Problem

• Robot Explainability
• Complex Environment
• Human-Robot Collaboration

Innovation

• Automated Behavioral Learning
• Online System with Human Feedback
• Robotics Operation and Training