Search papers, labs, and topics across Lattice.
This paper introduces a contextual semantic mapping pipeline that integrates SLAM-based geometric mapping with vision-language model (VLM) reasoning to enhance the semantic understanding of objects in intralogistics environments. By leveraging multi-view observations and zero-shot learning, the pipeline effectively infers contextual properties such as object movability without the need for task-specific training. The results indicate a high accuracy in semantic classification (98.93% mIoU) and object movability estimation (89.17% mAcc), highlighting the critical role of VLM reasoning in achieving contextual understanding.
Contextual semantic mapping can achieve near-perfect accuracy in intralogistics by inferring object properties without predefined categories or task-specific training.
Autonomous mobile robots operating in intralogistics environments rely on geometric maps for localization and navigation, but lack semantic understanding of objects and their contextual properties. We present a contextual semantic mapping pipeline that combines SLAM-based geometric mapping, SAM-based instance segmentation, instance clustering, and VLM multi-view reasoning to produce a contextual semantic map representation encoding geometric structure, object class, and object movability. By aggregating observations across multiple viewpoints and querying a VLM in a zero-shot, open-vocabulary setting, the pipeline infers contextual object properties--here demonstrated through movability--without requiring task-specific training or predefined object categories. We evaluate three VLMs under two prompting strategies and conduct a component-wise analysis of the pipeline. The proposed pipeline achieves 98.93 % mIoU for semantic classification and 89.17 % mAcc for object movability estimation. Component analysis identifies VLM reasoning as the primary bottleneck for contextual understanding and instance clustering as the main limitation for panoptic performance. The resulting semantic map supports context-aware filtering and robust navigation in dynamic intralogistics environments.
