Developing Models of Procedural Skills using an AI-assisted Text-to-Model Approach

2026-04-19

AI Summary

This work addresses the scalability bottleneck in AI tutoring systems caused by the labor-intensive, manual construction of structured procedural skill models. To overcome this limitation, the authors propose a human-in-the-loop text-to-model generation approach that leverages large language models to automatically transform instructional texts into procedural skill models conforming to the Task-Method-Knowledge (TMK) ontology. The method integrates ontology-constrained prompting with template-driven generation and incorporates expert validation of causal logic and failure conditions. This framework preserves model structural integrity and semantic alignment while substantially reducing expert modeling effort. Evaluated in a graduate-level AI course, the approach produced 23 skill models with 50–70% less expert time investment, and the generated models demonstrated high reproducibility under fixed inputs.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Structured Learning
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Scalable AI tutoring for procedural skill learning requires structured knowledge representations, yet constructing these representations remains a labor-intensive bottleneck. This paper presents a human-in-the-loop text-to-model pipeline that uses large language models to transform instructional materials into schema-complete Task-Method-Knowledge models of procedural skills through ontology-constrained prompting and template-based generation. The approach automates structural scaffolding while preserving expert oversight for validating causal transitions and failure conditions. We apply the pipeline to instructional materials from a graduate-level online AI course, constructing 23 procedural skill models. AI-assisted authoring reduced expert modeling time by 50-70% while producing structurally valid and highly reproducible models under fixed-input conditions. We evaluate structural validity, semantic alignment, reproducibility, and refinement effort to characterize authoring scalability. Results indicate that AI-assisted text-to-model methods can substantially lower the cost of constructing structured procedural representations, making course-wide deployment of structured AI coaching systems practically feasible.

Problem

• procedural skills
• structured knowledge representation
• AI tutoring
• modeling bottleneck
• scalability

Innovation

• text-to-model
• procedural skill modeling
• AI-assisted authoring
• ontology-constrained prompting
• structured knowledge representation