Search papers, labs, and topics across Lattice.
This paper investigates the distinction between content danger (CD) and physical danger (PD) in large language models (LLMs) when applied to embodied agents, revealing that these dangers are represented as separable signals in LLM hidden states. Utilizing a novel probing method called PRISM, the authors achieve high accuracy in detecting physical risks, outperforming existing LLM judges on a new benchmark, PhysicalSafetyBench-1K (PSB-1K). The findings underscore the inadequacy of traditional text-based safety measures and highlight the necessity for advanced methods to assess physical safety in AI systems.
LLMs can misinterpret benign text as physically dangerous actions, but a new probing method achieves over 99% accuracy in identifying these risks without relying on explicit unsafe keywords.
Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world. We study whether this physically grounded danger is the same safety problem as ordinary text-level content danger. Through hidden-state direction analysis and random-split null tests, we show that content danger (CD) and physical danger (PD) form separable signals in LLM representations across Qwen2.5-3B/7B/14B/32B, Phi-3.5 and SmolLM2. Building on the CD/PD separability, we propose PRISM, a single-layer L2-regularized logistic probe over full hidden states. PRISM achieves 86.2--87.7\% accuracy on SafeAgentBench with 11.7--13.7\% FPR, while same-scale LLM judges over-block safe tasks at 24.7--39.0\% FPR. We further introduce PhysicalSafetyBench-1K (PSB-1K), a contrastive benchmark of 1{,}000 physical-risk pairs without direct harm keywords, to test whether methods detect physically grounded danger rather than explicit unsafe wording. On PSB-1K, PRISM reaches 99.6\% accuracy and 0.7\% FPR, whereas a Qwen2.5-3B judge rejects 67.8\% of safe tasks. PRISM also replicates on SafeText and EARBench, supporting hidden-state probing as a representation-level method for physical safety beyond text moderation.
Mila
Tsinghua AI