Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces OptiSight, a hybrid framework that integrates Vision-Language Model (VLM) reasoning with deterministic visual servoing to enhance autonomous indoor navigation. By employing a finite-state Chain-of-Thought architecture, the system efficiently localizes targets using Grounded-SAM and translates visual inputs into navigation commands without the need for dense mapping. Experimental results in AI Habitat show that OptiSight achieves reliable zero-shot navigation across varied indoor environments while maintaining a low computational footprint of 8 GB VRAM, effectively addressing challenges like obstacle avoidance and semantic ambiguity.
Zero-shot navigation in complex indoor environments is now achievable with a mere 8 GB VRAM, thanks to a novel integration of semantic reasoning and geometric control.
Autonomous indoor navigation requires both semantic understanding and precise geometric control. We propose OptiSight, a hybrid framework that combines Vision-Language Model reasoning with deterministic visual servoing through a finite-state Chain-of-Thought architecture. Grounded-SAM localizes open-vocabulary targets, while camera projection geometry converts visual observations into navigation commands without requiring dense mapping. The VLM is queried only at key decision points, reducing computational overhead while geometric control handles continuous navigation. Experiments in AI Habitat demonstrate reliable zero-shot navigation across diverse indoor scenarios, including obstacle avoidance and semantic ambiguity, while operating within an 8~GB VRAM budget. The source code is available at https://github.com/avanalperen/OptiSight-Python-Multimodal-CoT-for-Visual-Reasoning.