Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces PyRUA-Lean, an interactive code-execution framework for VLM robot agents that improves task success by 14% while drastically reducing token usage. The framework achieves this by composing classical robot primitives with learned VLA policies, enabling conditional checks and local retries within Python cells, thereby minimizing redundant observations and model invocations. Evaluated on three benchmark datasets, PyRUA-Lean significantly outperforms a tool-calling baseline in terms of both success rate and efficiency under equal LLM-call budgets.
Robot agents achieve 14% higher success rates and use 65% fewer tokens by composing primitives and performing local retries within an interactive code execution framework.
Vision language model (VLM) agents can control robots through visual feedback and action primitives, but repeated model invocations and redundant observations incur substantial token overhead. We introduce PyRUA-Lean, an interactive code-execution framework that couples feedback-driven primitive composition with selective observation: the agent composes classical robot primitives and learned vision-language-action (VLA) policies into Python cells that perform conditional checks and local retries, returning only explicitly requested images and state feedback for replanning. Across 700 simulated task instances from LIBERO-PRO, RoboTwin 2.0, and RoboCasa365, we compare PyRUA-Lean with a tool-calling baseline using the same GPT-6 Astra planner and underlying robot primitives. Under equal LLM-call budgets, PyRUA-Lean increases overall success from 63.1% to 71.7%. On instances solved by both agents, it uses 49% fewer LLM calls and 65% fewer input tokens.