Search papers, labs, and topics across Lattice.
This paper introduces a "Map-Based AI" approach for ObjectNav, integrating LLM-based semantic inference with a hybrid topological-grid mapping system to overcome the limitations of reactive LLM agents. A fine-tuned Llama-2 model, using LoRA, infers semantic zone categories and target existence probabilities from object observations, which are then incorporated into a topological graph. Experiments in AI2-THOR show that this method significantly improves Success Rate and SPL compared to frontier exploration and reactive LLM baselines.
Ditch the reactive agent: a Llama-2 model fine-tuned to infer semantic zones from object observations enables systematic exploration via TSP optimization, dramatically boosting ObjectNav performance.
Object-Goal Navigation (ObjectNav) requires an agent to find and navigate to a target object category in unknown environments. While recent Large Language Model (LLM)-based agents exhibit zero-shot reasoning, they often rely on a "reactive" paradigm that lacks explicit spatial memory, leading to redundant exploration and myopic behaviors. To address these limitations, we propose a transition from reactive AI to "Map-Based AI" by integrating LLM-based semantic inference with a hybrid topological-grid mapping system. Our framework employs a fine-tuned Llama-2 model via Low-Rank Adaptation (LoRA) to infer semantic zone categories and target existence probabilities from verbalized object observations. In this study, a "zone" is defined as a functional area described by the set of observed objects, providing crucial semantic co-occurrence cues for finding the target. This semantic information is integrated into a topological graph, enabling the agent to prioritize high-probability areas and perform systematic exploration via Traveling Salesman Problem (TSP) optimization. Evaluations in the AI2-THOR simulator demonstrate that our approach significantly outperforms traditional frontier exploration and reactive LLM baselines, achieving a superior Success Rate (SR) and Success weighted by Path Length (SPL).
