Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the capabilities of GPT-5.1 as a high-level controller for a mobile robot, despite lacking prior embodiment or sensorimotor training. The model successfully navigated and performed object-directed tasks using only low-resolution images, demonstrating emergent spatial reasoning and physical understanding, such as maintaining memory of object locations and inferring the consequences of its actions. These findings challenge traditional views in cognitive science and robotics regarding the necessity of physical embodiment for intelligent behavior, suggesting that large language models can exhibit world-model-like behavior in embodied contexts.
GPT-5.1 navigates and interacts with physical environments without prior embodiment training, hinting at a surprising capacity for spatial reasoning and physical understanding.
This exploratory study examines whether a large multimodal language model, GPT-5.1, can serve as the high-level controller of a physical mobile robot despite having no prior embodiment, no training in simulated environments, and no exposure to sensorimotor experience. Using only low-resolution first-person images and a discrete action set, the model was tasked with navigation and object-directed behaviors such as locating and contacting a target toy. Across multiple trials, GPT-5.1 demonstrated emergent capabilities that suggest elements of spatial reasoning and physical understanding. These included maintaining short-term memory of object locations after they left the camera frame, inferring the physical consequences of its own movements, and executing coherent action sequences such as colliding with an object and reversing to visually verify the outcome. At the same time, the model displayed inefficiencies and perceptual limitations, including imprecise alignment strategies and occasional misidentification of distant distractors. Overall, the results indicate that GPT-5.1 exhibits signs of world-model-like behavior in an embodied setting, despite the absence of any embodiment-related training, a finding that challenges long-standing views in cognitive science and robotics which hold that a physical body is a necessary prerequisite for developing such forms of intelligence. The findings motivate deeper investigation into the emergence, limits, and robustness of physical understanding in large language models.