SHARPIE: A Modular Framework for Reinforcement Learning and Human-AI Interaction Experiments

2025-01-31

AI Summary

Existing research lacks a unified, flexible experimental platform for human-AI collaborative learning. Method: This paper proposes and implements an open-source, modular platform enabling systematic experimentation with humans and reinforcement learning (RL) agents in interactive tasks. It integrates standardized environment wrappers (compatible with OpenAI Gym), RL algorithm libraries (PyTorch/TensorFlow), a web interface (Flask), structured logging, cloud deployment (AWS/Heroku), and interfaces with crowdsourcing platforms (Prolific/MTurk). Contribution/Results: The platform introduces the first general-purpose human-AI RL interaction interface and standardizes experimental paradigms. Its modular architecture significantly enhances cross-task reusability and extensibility. Deployed in over ten human-AI collaboration studies, it reduces average experiment setup time by 70%. The platform has been adopted as standard experimental infrastructure by five universities and two AI ethics laboratories.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Intelligent Robots: Human-Robot Interaction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Reinforcement learning (RL) offers a general approach for modeling and training AI agents, including human-AI interaction scenarios. In this paper, we propose SHARPIE (Shared Human-AI Reinforcement Learning Platform for Interactive Experiments) to address the need for a generic framework to support experiments with RL agents and humans. Its modular design consists of a versatile wrapper for RL environments and algorithm libraries, a participant-facing web interface, logging utilities, deployment on popular cloud and participant recruitment platforms. It empowers researchers to study a wide variety of research questions related to the interaction between humans and RL agents, including those related to interactive reward specification and learning, learning from human feedback, action delegation, preference elicitation, user-modeling, and human-AI teaming. The platform is based on a generic interface for human-RL interactions that aims to standardize the field of study on RL in human contexts.

Problem

• Artificial Intelligence
• Human-AI Interaction
• Learning Efficiency

Innovation

• Modular Reinforcement Learning Platform
• Human-AI Collaboration
• Standardized Experimental Framework