InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersons

2026-02-04

AI Summary

This work addresses the high barrier to entry in existing imitation learning methods, which typically require extensive expert demonstrations and continuous monitoring, rendering them inaccessible to non-expert users. To overcome this limitation, we propose an interactive policy restructuring and training framework that enables ordinary users without machine learning expertise to dynamically adjust an agent’s policy structure and parameters through natural-language instructions and a minimal number of demonstrations. Our approach integrates instruction parsing, dynamic policy architecture evolution, online imitation learning, and human-in-the-loop interaction mechanisms. A user study (N=34) demonstrates that, compared to baseline methods, the proposed framework significantly enhances policy robustness and trustworthiness while maintaining high usability.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Intelligent User Interfaces
• Intelligent Robots: Human-Robot Interaction

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Imitation learning has shown success in many tasks by learning from expert demonstrations. However, most existing work relies on large-scale demonstrations from technical professionals and close monitoring of the training process. These are challenging for a layperson when they want to teach the agent new skills. To lower the barrier of teaching AI agents, we propose Interactive Policy Restructuring and Training (InterPReT), which takes user instructions to continually update the policy structure and optimize its parameters to fit user demonstrations. This enables end-users to interactively give instructions and demonstrations, monitor the agent's performance, and review the agent's decision-making strategies. A user study (N=34) on teaching an AI agent to drive in a racing game confirms that our approach yields more robust policies without impairing system usability, compared to a generic imitation learning baseline, when a layperson is responsible for both giving demonstrations and determining when to stop. This shows that our method is more suitable for end-users without much technical background in machine learning to train a dependable policy

Problem

• imitation learning
• layperson
• interactive teaching
• policy learning
• end-user AI training

Innovation

• Imitation Learning
• Interactive Policy Restructuring
• End-user Teaching
• Layperson Demonstration
• Policy Optimization