Search papers, labs, and topics across Lattice.
Vesta is a unified embodied reasoning model that integrates localization, spatial reasoning, navigation, and long-horizon planning into a single foundation model, addressing the inefficiencies of multi-model stacks. By leveraging a large curated corpus for spatial grounding and a multimodal memory system, Vesta achieves significant performance improvements, surpassing individual state-of-the-art (SOTA) models by over 20% and outperforming ensembles of top-performing specialists by more than 10%. In practical applications, Vesta enhances task success rates in real-world robotic scenarios by over 35%, showcasing the advantages of a generalist approach in complex environments.
A single generalist model outperforms specialized systems, achieving over 35% improvement in real-world robotic task success.
Robots operating in open-world environments must seamlessly integrate localization, spatial reasoning, navigation, and long-horizon planning. While specialist models excel at individual tasks, deploying a multi-model stack is computationally expensive and prone to cascading errors. We present Vesta, a unified embodied generalist that consolidates these capabilities into a single foundation model. Our approach combines a diverse and massive curated corpus designed to induce spatial grounding and a simple multimodal memory harness that enables reasoning over extended time horizons. Across diverse benchmarks, Vesta on average beats individual SOTA baselines by >20% and beats an ensemble of per-category-best baselines by >10% -- thus demonstrating that a generalist model can match or exceed specialists. On real-world robotic tasks requiring memory and reasoning, Vesta improves task success by >35\%. Our work thus demonstrates that a single generalist is a feasible, scalable, and arguably preferable alternative to combining specialists.
BAIR
NVIDIA