Search papers, labs, and topics across Lattice.
The paper introduces SafetyALFRED, a benchmark extending ALFRED with six categories of real-world kitchen hazards, to evaluate the safety-conscious planning abilities of multimodal LLMs. They tested eleven state-of-the-art models (Qwen, Gemma, Gemini) on hazard recognition and active risk mitigation in embodied environments, revealing a significant gap between performance on static QA-based hazard recognition and actual mitigation success. The results highlight the inadequacy of static QA evaluations for assessing physical safety in embodied agents.
LLMs that ace hazard recognition in question answering settings still fail to mitigate those same hazards when embodied in a simulated kitchen environment.
Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insufficient. We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards. While existing safety evaluations focus on hazard recognition through disembodied question answering (QA) settings, we evaluate eleven state-of-the-art models from the Qwen, Gemma, and Gemini families on not only hazard recognition, but also active risk mitigation through embodied planning. Our experimental results reveal a significant alignment gap: while models can accurately recognize hazards in QA settings, average mitigation success rates for these hazards are low in comparison. Our findings demonstrate that static evaluations through QA are insufficient for physical safety, thus we advocate for a paradigm shift toward benchmarks that prioritize corrective actions in embodied contexts. We open-source our code and dataset under https://github.com/sled-group/SafetyALFRED.git
