Search papers, labs, and topics across Lattice.
This paper introduces ToolHazard, a novel framework designed to synthesize scalable adversarial environments for evaluating the security and alignment of LLM-based agents. By automating the generation of executable stateful environments and identifying viable injection points, ToolHazard enables extensive testing of agents against a variety of environmental attacks without the need for extensive manual engineering. Experiments demonstrate that the framework not only uncovers significant vulnerabilities in agent behavior but also enhances alignment data quality, improving security while maintaining task performance.
ToolHazard reveals that injection timing and placement are critical factors in exploiting vulnerabilities of LLM-based agents, leading to new insights in adversarial robustness.
Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose **ToolHazard**, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build **ToolHazard-Bench** for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.
