Search papers, labs, and topics across Lattice.
This paper introduces ROSA, a Robotics Foundation Model serving system designed to enhance the efficiency of robot factories by leveraging shared GPU resources and multi-model pipelines. By shifting the focus from minimizing individual action latency to maximizing overall factory productivity, ROSA employs a factory-objective-driven scheduling approach that significantly improves performance metrics. Evaluation results indicate that ROSA can boost factory productivity by up to 12.06 times compared to traditional dedicated serving systems, marking a substantial advancement in robotics deployment in industrial settings.
ROSA revolutionizes robot factory operations by boosting productivity up to 12.06x through innovative shared GPU-pool serving and factory-focused scheduling.
Robotics foundation models (RFMs) are making general-purpose robots increasingly practical for factory deployments. While RFM serving systems are central to this vision, existing systems are largely shaped by a single-robot, single-model assumption: inference is treated as an edge-computing problem handled by an on-robot or dedicated nearby GPU, and the serving objective is to minimize the latency of a single action model. In this paper, we propose ROSA, an RFM serving system for robot factories designed around three key principles. First, ROSA adopts shared GPU-pool serving, allowing a fleet of robots to access powerful server-class GPUs over the network in order to improve inference performance, battery duration, and GPU utilization. Second, ROSA provides a robotics-aware programming abstraction and system design that supports multi-model pipelines, per-task performance requirements, and failure handling. Third, ROSA uses factory-objective-driven scheduling to maximize SLO-qualified factory productivity rather than minimizing individual request latency. We implement ROSA on top of Ray Serve for distributed orchestration, with vLLM, PyTorch, and JAX as model-serving backends, and evaluate it on both real robots and synthetic large-scale workloads. The results show that ROSA improves factory productivity by up to 12.06x over conventional dedicated serving systems.
NVIDIA