Search papers, labs, and topics across Lattice.
This paper introduces event-adaptive motion planning (EAMP), a novel framework that enhances robot navigation in safety-critical environments by integrating a distilled vision-language model (VLM) to address unforeseen semantic events. By employing a prompt-configurable semantic event trigger (PC-SET), the system monitors for behavioral anomalies and activates a distilled SemNav-VLM to generate strategy-level decisions. Experimental results show that EAMP significantly improves dynamic safety margins while maintaining real-time efficiency compared to existing methods, highlighting its potential for safer robotic operations in complex scenarios.
EAMP boosts robotic navigation safety by dynamically adapting to unforeseen events, achieving superior performance without sacrificing real-time responsiveness.
Robot navigation in safety-critical scenarios faces significant challenges from unforeseen semantic events, where collisions arise primarily from the unpredictable behaviors of dynamic agents rather than unseen objects. While large vision-language models (VLMs) offer remarkable capabilities in commonsense reasoning, frequently invoking them within the continuous control loop introduces severe computational latency, fundamentally destabilizing physical execution. To address these challenges, we propose event-adaptive motion planning (EAMP), an efficient framework for VLM-based robot navigation. Specifically, a prompt-configurable semantic event trigger (PC-SET) selectively activates semantic intervention by continuously monitoring short temporal clips for behavioral anomalies. Upon triggering, an event-triggered distilled SemNav-VLM, fine-tuned via physically verified semantic distillation, maps detected anomalies into discrete strategy-level decisions. Subsequently, a semantic model predictive control (SMPC) module translates these strategies into dynamic reconfigurations of optimization objectives and geometric references. Extensive experiments in safety-critical logistics scenarios demonstrate that EAMP effectively aligns high-level reasoning with low-level control, significantly improving dynamic safety margins over existing baselines while preserving real-time efficiency.
