Decentralized LLM-Driven Coordination of Acoustic Robots for Contactless Object Manipulation

2026-05-28

AI Summary

This study addresses the challenge of enabling non-expert users to efficiently control distributed acoustic robots for contactless object manipulation through natural language. The work proposes the first decentralized framework integrating large language models (LLMs) with distributed acoustic robotic systems. In this approach, user speech is transcribed via Whisper and parsed by an LLM into a structured JSON task plan encoding spatiotemporal constraints and synchronization requirements. This plan is then executed collaboratively by multiple mobile robots equipped with ultrasonic phased arrays, supporting sequential, parallel, and synchronized manipulation modes. Experimental evaluations across three task categories demonstrate success rates of 96%, 86%, and 70%, respectively, validating the feasibility and effectiveness of this end-to-end, natural language–driven paradigm for acoustic manipulation.

Abstract

Natural language interfaces can simplify interaction with multi-robot systems, especially when non-expert users need to issue high-level commands. Acoustic manipulation using ultrasonic phased arrays also enables contactless object handling for applications such as healthcare, laboratory automation, and precision transport. However, combining large language models (LLMs) with distributed acoustic mobile robots remains underexplored. This paper presents a decentralized framework for natural language-driven coordination of acoustic robots for contactless object manipulation. The system converts spoken instructions into executable multi-robot task plans using Whisper-based speech recognition, LLM-based semantic parsing, structured JSON task representation, and distributed scheduling. The JSON schema encodes robot assignments, temporal dependencies, spatial constraints, and synchronization requirements for sequential, parallel, and synchronized execution. The system is implemented on two TurtleBot3-based acoustic robots, each equipped with an ultrasonic phased array for contactless object transport. Experiments were conducted in three scenarios: sequential execution, parallel multi-robot transport, and synchronized cooperative manipulation. The system achieved task success rates of 96 percent for sequential tasks, 86 percent for parallel execution, and 70 percent for synchronized collaborative transport. These results show that natural language commands can be transformed into distributed robot actions for contactless manipulation, highlighting the potential of LLM-driven automation for human-robot interaction in distributed robotic systems.

Problem

• decentralized coordination
• large language models
• acoustic manipulation
• contactless object manipulation
• multi-robot systems

Innovation

• decentralized coordination
• large language models (LLMs)
• acoustic manipulation
• natural language interface
• ultrasonic phased array