Effect of Adaptive Communication Support on LLM-powered Human-Robot Collaboration

2024-11-26

AI Summary

This paper addresses the lack of dynamic adaptability in robot support strategies for human-robot collaboration. We propose HRT-ML, a framework that leverages LLM-driven multimodal language feedback to adaptively modulate the robot’s role (Coordinator or Manager) and support frequency based on real-time task complexity and human capability alignment. We introduce the principle of communication adaptivity and empirically demonstrate that “over-proactive feedback”—beyond the LLM’s capability boundary—introduces noise, exacerbates human cognitive load, and degrades team performance. Evaluated in an enhanced Overcooked simulation environment, our approach shows that proactive, high-frequency support significantly improves human preference and collaboration efficiency for medium-to-high-complexity tasks; however, it fails when task demands exceed the LLM’s reasoning capacity. Our core contributions are: (1) a task- and capability-driven dynamic role-switching mechanism, and (2) a generalizable adaptive feedback scheduling paradigm grounded in human-AI co-adaptation principles.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Intelligent Robots: Human-Robot Interaction
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Effective human-robot collaboration requires robot to adopt their roles and levels of support based on human needs, task requirements, and complexity. Traditional human-robot teaming often relies on a pre-determined robot communication scheme, restricting teamwork adaptability in complex tasks. Leveraging strong communication capabilities of Large Language Models (LLMs), we propose a Human-Robot Teaming Framework with Multi-Modal Language feedback (HRT-ML), a framework designed to enhance human-robot interaction by adjusting the frequency and content of language-based feedback. HRT-ML framework includes two core modules: a Coordinator for high-level, low-frequency strategic guidance, and a Manager for subtask-specific, high-frequency instructions, enabling passive and active interactions with human teammates. To assess the impact of language feedback in collaborative scenarios, we conducted experiments in an enhanced Overcooked environment with varying levels of task complexity (easy, medium, hard) and feedback frequency (inactive, passive, active, superactive). Our results show that as task complexity increases relative to human capabilities, human teammates exhibited a stronger preference towards robotic agents that can offer frequent, proactive support. However, when task complexities exceed the LLM's capacity, noisy and inaccurate feedback from superactive robotic agents can instead hinder team performance, as it requires human teammates to increase their effort to interpret and respond to a large number of communications, with limited performance return. Our results offer a general principle for robotic agents to dynamically adjust their levels and frequencies of communications to work seamlessly with humans and achieve improved teaming performance.

Problem

• Enhance human-robot collaboration adaptability
• Adjust feedback frequency and content dynamically
• Optimize team performance through adaptive communication

Innovation

• Adaptive communication support
• Multi-Modal Language feedback
• Dynamic adjustment of feedback