🤖 AI Summary
This work addresses the challenge of enabling robots in open-world environments to simultaneously perform localization, spatial reasoning, navigation, and long-horizon planning. To overcome the computational overhead and error propagation inherent in stacking multiple specialized models, the authors propose the first unified embodied generalist model. Trained on a large-scale spatially grounded corpus and augmented with a lightweight multimodal memory mechanism, the model supports end-to-end reasoning for complex tasks and long-term planning. Experiments demonstrate that it surpasses single-task state-of-the-art models by over 20% on average and outperforms the best expert ensembles by more than 10% across multiple benchmarks. Notably, it achieves a greater than 35% improvement in success rate on real-world robotic tasks, establishing for the first time that a single generalist model can comprehensively exceed combinations of specialized models while offering superior scalability and deployment efficiency.
📝 Abstract
Robots operating in open-world environments must seamlessly integrate localization, spatial reasoning, navigation, and long-horizon planning. While specialist models excel at individual tasks, deploying a multi-model stack is computationally expensive and prone to cascading errors. We present Vesta, a unified embodied generalist that consolidates these capabilities into a single foundation model. Our approach combines a diverse and massive curated corpus designed to induce spatial grounding and a simple multimodal memory harness that enables reasoning over extended time horizons. Across diverse benchmarks, Vesta on average beats individual SOTA baselines by >$20\%$ and beats an ensemble of per-category-best baselines by $>10\%$ -- thus demonstrating that a generalist model can match or exceed specialists. On real-world robotic tasks requiring memory and reasoning, Vesta improves task success by >35\%. Our work thus demonstrates that a single generalist is a feasible, scalable, and arguably preferable alternative to combining specialists.