Search papers, labs, and topics across Lattice.
This paper introduces LangLoc, a novel three-stage pipeline for fine-grained indoor localization using natural language descriptions to estimate an observer's 2D position and heading within a known 3D environment. By employing a dual-branch GATv2 encoder with CLIP semantic features, LangLoc achieves an 8 percentage point improvement in Top-1 recall for scene retrieval, while also employing a ray-cast object visibility method to estimate position with a median error of just 0.95 meters. The system further refines its estimates through a Bayesian dialog module that interacts with users to resolve ambiguities, supported by a comprehensive benchmark of over 13,000 pose-indexed descriptions across 1,300 indoor scans.
LangLoc achieves unprecedented accuracy in indoor localization from natural language, closing the gap between coarse scene retrieval and precise pose estimation.
We tackle fine-grained indoor localization from natural language: given a free-form description of one's surroundings, estimate the observer's 2D position and heading within a known 3D environment. Language queries are lightweight, privacy-preserving, and need no camera - yet prior work stops at coarse scene retrieval and cannot resolve an intra-scene pose. We close this gap with LangLoc, a three-stage pipeline that (i) retrieves the correct scene via a dual-branch GATv2 encoder with CLIP semantic features, surpassing the previous best by 8 percentage points in Top-1 recall; (ii) estimates position and heading by scoring a dense floor grid through ray-cast object visibility, reaching a median error of 0.95 m; and (iii) resolves residual ambiguity through a Bayesian dialog module that asks targeted yes/no questions and updates a pose posterior until the location is pinpointed. To support this task we contribute a benchmark of $13{,}000{+}$ pose-indexed natural-language descriptions over $1{,}300{+}$ indoor 3D scans.
ETH
Microsoft Research