Search papers, labs, and topics across Lattice.
This paper presents a measurement study of mobile robotic manipulation workloads, evaluating the feasibility and trade-offs of running them on onboard, edge, and cloud GPU platforms. The study finds that onboard GPUs struggle with the full workload stack or drain batteries quickly, while offloading introduces latency and bandwidth challenges that impact task accuracy. They quantify the performance and resource utilization of different compute platforms, highlighting the challenges of balancing compute location with task performance and resource constraints.
Running robotic manipulation workloads entirely onboard kills robot batteries, but offloading to the cloud tanks accuracy due to network latency, revealing a critical compute placement trade-off.
Mobile robotic manipulation--the ability of robots to navigate spaces and interact with objects--is a core capability of physical AI. Foundation models have led to breakthroughs in their performance, but at a significant computational cost. We present the first measurement study of mobile robotic manipulation workloads across onboard, edge, and cloud GPU platforms. We find that the full workload stack is infeasible to run on smaller onboard GPUs, while larger onboard GPUs drain robot batteries several hours faster. Offloading alleviates these constraints but introduces its own challenges, as additional network latency degrades task accuracy, and the bandwidth requirement makes naive cloud offloading impractical. Finally, we quantify opportunities and pitfalls of sharing compute across robot fleets. We believe our measurement study will be crucial to designing inference systems for mobile robots.
BAIR
Microsoft Research