🤖 AI Summary
This work addresses the challenges of deploying advanced AI models on robots, which are often constrained by high computational overhead, latency, and energy consumption. The authors propose a modular, declarative, and model-agnostic multimodal inference framework that enables flexible composition of models, hardware, and data pipelines through lightweight YAML configurations, supporting edge offloading without code modifications. Built upon ROS 2, the system integrates components such as automatic speech recognition (ASR), large language models (LLMs), and convolutional neural networks (CNNs), leveraging I/O adapters, pluggable inference backends, and post-processing logic to ensure modularity and reproducibility. Experimental results demonstrate that, compared to fully onboard execution, the edge-deployed configuration reduces on-robot computational utilization by 83.16% and decreases energy consumption by 65.8%.
📝 Abstract
Recent advances in artificial intelligence (AI) have enabled effective perception and language models for robots, but their deployment remains computationally expensive, increasing latency and energy use. This work presents the Open Robotics Inference and Control Framework (ORICF), a modular, declarative, and model-agnostic platform for composing multimodal robotic inference pipelines. ORICF integrates input/output (I/O) adapters, pluggable inference back ends, and post-processing logic, while lightweight YAML specifications allow models, hardware targets, and data channels to be changed without code modification. The framework also supports edge offloading, i.e., executing inference on nearby external computers instead of onboard the robot. ORICF is evaluated on a mobile robot that answers spoken queries about people detected in its camera stream by combining automatic speech recognition (ASR), a large language model (LLM), and a convolutional neural network (CNN) detector through Robot Operating System 2 (ROS2). Compared with onboard execution, ORICF-based edge deployment reduces robot-side compute utilization by up to 83.16% and estimated energy consumption by 65.8%, while preserving modularity and reproducibility.