ROSA: A Robotics Foundation Model Serving System for Robot Factories

2026-07-01

AI Summary

This work addresses the limitations of existing foundation model serving systems for robotics, which are typically designed under single-robot, single-model assumptions and thus fail to meet the demands of factory-scale multi-robot collaboration in terms of performance, resource efficiency, and task diversity. To overcome these challenges, we propose a foundation model serving system tailored for robotic factories, introducing a shared GPU pool architecture, a robot-aware programming abstraction, and a factory-level service-level objective (SLO)-driven scheduling policy that departs from conventional request-latency-centric paradigms. Built on Ray Serve, our system integrates vLLM, PyTorch, and JAX backends to enable networked GPU resource sharing and multi-model inference pipelines. Experiments under real robotic deployments and large-scale synthetic workloads demonstrate up to a 12.06× improvement in factory-level productivity.

Abstract

Robotics foundation models (RFMs) are making general-purpose robots increasingly practical for factory deployments. While RFM serving systems are central to this vision, existing systems are largely shaped by a single-robot, single-model assumption: inference is treated as an edge-computing problem handled by an on-robot or dedicated nearby GPU, and the serving objective is to minimize the latency of a single action model. In this paper, we propose ROSA, an RFM serving system for robot factories designed around three key principles. First, ROSA adopts shared GPU-pool serving, allowing a fleet of robots to access powerful server-class GPUs over the network in order to improve inference performance, battery duration, and GPU utilization. Second, ROSA provides a robotics-aware programming abstraction and system design that supports multi-model pipelines, per-task performance requirements, and failure handling. Third, ROSA uses factory-objective-driven scheduling to maximize SLO-qualified factory productivity rather than minimizing individual request latency. We implement ROSA on top of Ray Serve for distributed orchestration, with vLLM, PyTorch, and JAX as model-serving backends, and evaluate it on both real robots and synthetic large-scale workloads. The results show that ROSA improves factory productivity by up to 12.06x over conventional dedicated serving systems.

Problem

• robotics foundation models
• RFM serving system
• robot factories
• multi-model pipelines
• factory productivity

Innovation

• Robotics Foundation Models
• Shared GPU Pool
• Multi-model Pipeline
• Factory-level Scheduling
• SLO-aware Serving