Search papers, labs, and topics across Lattice.
This paper introduces a semantic zone-based map management approach for mobile robots to address memory constraints when deploying both large AI models and dense maps. The method associates keyframes with semantic indoor regions, prioritizing spatially relevant map content at the semantic zone level to reduce keyframe loading/unloading frequency and memory usage. Experiments in simulated environments and on an NVIDIA Jetson Orin Nano show that the proposed approach improves throughput by 3.3 tokens/s and reduces latency by 21.7% compared to a geometric map-management strategy, while also eliminating out-of-memory failures.
Semantic scene understanding can keep your robot from crashing when running LLMs on edge devices.
Recent advances in large AI models (VLMs and LLMs) and joint use of the 3D dense maps, enable mobile robots to provide more powerful and interactive services grounded in rich spatial context. However, deploying both heavy AI models and dense maps on edge robots is challenging under strict memory budgets. When the memory budget is exceeded, required keyframes may not be loaded in time, which can degrade the stability of position estimation and interfering model performance. We proposes a semantic zone-based map management approach to stabilize dense-map utilization under memory constraints. We associate keyframes with semantic indoor regions (e.g., rooms and corridors) and keyframe management at the semantic zone level prioritizes spatially relevant map content while respecting memory constraints. This reduces keyframe loading and unloading frequency and memory usage. We evaluate the proposed approach in large-scale simulated indoor environments and on an NVIDIA Jetson Orin Nano under concurrent SLAM-VLM execution. With Qwen3.5:0.8b, the proposed method improves throughput by 3.3 tokens/s and reduces latency by 21.7% relative to a geometric map-management strategy. Furthermore, while the geometric strategy suffers from out-of-memory failures and stalled execution under memory pressure, the proposed method eliminates both issues, preserving localization stability and enabling robust VLM operation. These results demonstrate that the proposed approach enables efficient dense map utilization for memory constrained, AI-integrated mobile robots. Code is available at: https://github.com/huichangs/rtabmap/tree/segment
