Enhanced Question-Answering for Skill-based learning using Knowledge-based AI and Generative AI

2025-04-10

AI Summary

Existing practice systems and chat agents in online skill learning inadequately support learners’ acquisition of procedural knowledge (“how to do”) and causal reasoning (“why it works”). Method: We propose TMK-LLM, the first framework integrating the Task–Method–Knowledge (TMK) structured knowledge model with large language models (LLMs). It employs knowledge-graph-driven TMK modeling, iterative prompt refinement, and multi-principle constrained generation to produce explanations that are goal-oriented, causally grounded, and compositionally coherent. Contribution/Results: Experiments demonstrate that TMK-LLM significantly outperforms baselines in explanation depth and relevance. It enables learners to construct comprehensive skill cognitive models by explicitly linking tasks, executable methods, and underlying domain knowledge. The framework establishes a novel, explainability-enhanced paradigm for intelligent tutoring agents—bridging the gap between LLM-generated responses and pedagogically meaningful, structured skill understanding.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Supporting learners' understanding of taught skills in online settings is a longstanding challenge. While exercises and chat-based agents can evaluate understanding in limited contexts, this challenge is magnified when learners seek explanations that delve into procedural knowledge (how things are done) and reasoning (why things happen). We hypothesize that an intelligent agent's ability to understand and explain learners' questions about skills can be significantly enhanced using the TMK (Task-Method-Knowledge) model, a Knowledge-based AI framework. We introduce Ivy, an intelligent agent that leverages an LLM and iterative refinement techniques to generate explanations that embody teleological, causal, and compositional principles. Our initial evaluation demonstrates that this approach goes beyond the typical shallow responses produced by an agent with access to unstructured text, thereby substantially improving the depth and relevance of feedback. This can potentially ensure learners develop a comprehensive understanding of skills crucial for effective problem-solving in online environments.

Problem

• Enhancing skill-based learning through AI-driven explanations
• Improving procedural and reasoning understanding in online education
• Generating deep, relevant feedback using Knowledge-based AI and LLMs

Innovation

• Knowledge-based AI framework TMK model
• LLM with iterative refinement techniques
• Teleological causal compositional explanations