Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To map the operational boundaries between monolithic multimodal architectures and task-specific vision pipelines, this study evaluates GPT-6 Astra and five other frontier general-purpose models against dedicated systems and human baselines across 55 benchmarks spanning 34 capabilities. The evaluation reveals that while Astra achieves state-of-the-art gains in visual reasoning, semantic interpretation, and object-centric prediction, all frontier generalist models consistently degrade on tasks requiring metric geometric accuracy, dense temporal consistency, and faithful visual reconstruction. These findings delineate a clear taxonomy between vision tasks that yield to generalist semantic scale and those that fundamentally demand specialized inductive biases or external tooling.
Frontier generalist systems have effectively mastered high-level visual semantics and spatial reasoning, yet they still stubbornly fail to displace dedicated models on metric geometry, dense temporal tracking, and fine-grained visual fidelity.
Frontier general-purpose systems are rapidly expanding beyond visual understanding into capabilities traditionally handled by dedicated computer-vision models. As these capabilities expand, a central question for the computer-vision community is how far this reach extends, and what remains hard. We evaluate GPT-6 Astra alongside five frontier general-purpose AI systems across 34 capabilities and 55 benchmarks spanning nine areas of computer vision. We compare their performance with dedicated models and humans where suitable references are available. Astra demonstrates broad visual capability, with substantial gains over other frontier systems in visual and spatial reasoning and several forms of structured prediction. Across the state-of-the-art systems, a consistent pattern emerges. Capabilities involving semantic interpretation, reasoning, and object-centric prediction increasingly approach or reach available reference levels. In contrast, larger gaps remain when tasks require metric geometric accuracy, faithful reconstruction, temporally consistent dense prediction, or specialized fine-grained visual knowledge. Additional reasoning and specialist tools close selected gaps, but their benefits vary across capabilities. These results map a changing landscape of computer vision in which increasingly sophisticated visual tasks are accessible through a general-purpose interface, while precise and fidelity-sensitive perception remains an important frontier.