Search papers, labs, and topics across Lattice.
This study explores agentic embodied control in autonomous agents, evaluating their performance in zero-shot vision-and-language navigation using minimal interfaces. The researchers found that their agent configurations achieved a success rate of 70.7% to 78% under varying conditions, demonstrating that even with limited resources, these agents can rival industrial-scale policies. Notably, the hybrid fable-5 agent, when equipped with a trained waypoint tool, significantly improved efficiency, achieving high success with fewer environment steps and reduced wall time compared to traditional methods.
Agentic control in zero-shot navigation rivals industrial-scale policies, achieving up to 78% success with minimal interfaces and fewer resources.
Autonomous embodied agents must sustain a long decision-making loop that involves perceiving, acting, verifying, and self-correcting over many steps. Current systems sustain this loop through task-specific workflows or embodied policies. We study a third form, agentic embodied control, in which a general-purpose agent holds the loop itself. Using zero-shot navigation as a controlled testbed, we evaluate three software-engineering agent harnesses given only a monocular RGB camera and discrete actions. Under this strictly minimal condition, replicated default-effort configurations reach 70.7$\pm$3.5% success (opus-5, mean over three runs), and fable-5 reaches 78% at maximum effort. When a trained waypoint tool is exposed alongside primitives as an optional capability, the hybrid fable-5 agent reaches 76.7$\pm$0.6% at default effort, using half the environment steps and less than one quarter of the wall time of the maximum-effort primitive run. Controlled interventions show that capability is primarily model-centered: model choice strongly changes success, harness effects are descriptive, and a forced waypoint interface helps weaker models but can hinder stronger ones. Performance nevertheless falls sharply on longer-horizon tasks, while latency and context growth limit sustained operation. These results show that agentic control is already competitive in zero-shot navigation and that models, harnesses, and interfaces offer complementary paths toward autonomous embodied agents.
