Search papers, labs, and topics across Lattice.
This study empirically evaluates frontier vision-language models (VLMs) as generalist robot agents using the Embodied Agent Arena, a benchmark comprising 1,000 cases across geometry, spatial reasoning, affordance, task planning, and manipulation. The arena isolates metric precision, functional grounding, and native goal completion to analyze VLM performance on robotic tasks. While VLMs show strength in precise estimation and localization, achieving coordinated, goal-directed actions remains a significant gap for robot generalism.
Frontier VLMs excel at precise scene estimation but falter in executing coordinated, goal-directed actions, highlighting a critical bottleneck for robot generalism.
Frontier vision-language models (VLMs) combine scene estimation, interaction grounding, and executable actions. Understanding how these abilities support complete robotic tasks is central to evaluating their readiness as robot generalists. We introduce Embodied Agent Arena to examine where local competence supports, or falls short of, complete task success across Geometry, Spatial Reasoning, Affordance, Task Planning, and Manipulation. The arena contains 1,000 cases drawn from 32 established sources and GeoProbe, our new benchmark for geometric estimation on Blender renders and real-scene images. A minimal harness preserves source observations and operations while separating metric precision, functional grounding, and native goal completion. We evaluate seven VLMs, analyze Astra's task-specific advantages, and compare richer-observation execution protocols and multi-round review. Across the arena, Astra's advantage is strongest in precise estimation and usable-contact localization; completing coordinated, goal-directed actions remains the key gap to robot generalism.
