Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a benchmarking framework to evaluate Vision-Language Models (VLMs) in crisis communication, focusing on guiding civilians during simulated evacuations. The study tests various communication strategies, environment representations, and threat behaviors, revealing that Narrowcast communication significantly reduces civilian Fail rates compared to Broadcast across diverse scenarios. Key findings indicate that the visual representation of environments enhances performance, while the introduction of adjacency graphs can be detrimental, especially in the presence of moving threats that complicate communication dynamics.
Narrowcast communication can cut civilian Fail rates in crisis evacuations, but the choice of environment representation can make or break the intervention's success.
Effective crisis response requires spatially grounded communication that bridges linguistic guidance of civilians with the physical environment, accounting for structural bottlenecks, evolving threats, and agent-specific contexts. Yet, current NLP research in crisis communication remains mainly limited to static, text-only classification settings, overlooking the critical communicative role of AI operators in dynamic, embodied scenarios. We address this gap with a novel benchmarking framework for evaluating Vision-Language Models (VLMs) tasked with guiding civilian agents through simulated evacuations. We test two communication strategies (narrowcast vs. broadcast), two environment representations (visual vs. graph-based), and two threat behaviors (static vs. moving) across nine maps of varying structural complexity. Our results show that Narrowcast consistently reduces civilian Fail rates compared to Broadcast across all difficulty levels. Guidance quality depends heavily on how the VLM operator represents the world: the visual modality drives performance, while adding an adjacency graph is model-dependent and often harmful. Moving threats raise Fail rates across all conditions as communication must continuously adapt over time. Together, these findings show that deploying VLMs as AI operators in evacuation scenarios remains a non-trivial challenge, where the choice of communication strategy and input representation can directly determine the success or failure of the intervention.