Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper presents a trajectory-based safety audit of Clawdbot, a tool-using AI agent, across six risk dimensions using a combination of adapted agent-safety benchmarks and hand-designed scenarios. The study logs complete interaction trajectories and evaluates safety using both an automated trajectory judge (AgentDoG-Qwen3-4B) and human review. The results reveal a non-uniform safety profile, with failures concentrated in scenarios involving underspecified intent, open-ended goals, or benign jailbreak prompts, leading to potentially high-impact tool actions.
Tool-using agents like Clawdbot are surprisingly vulnerable to seemingly harmless prompts, where minor misinterpretations can quickly escalate into high-stakes tool actions.
Clawdbot is a self-hosted, tool-using personal AI agent with a broad action space spanning local execution and web-mediated workflows, which raises heightened safety and security concerns under ambiguity and adversarial steering. We present a trajectory-centric evaluation of Clawdbot across six risk dimensions. Our test suite samples and lightly adapts scenarios from prior agent-safety benchmarks (including ATBench and LPS-Bench) and supplements them with hand-designed cases tailored to Clawdbot's tool surface. We log complete interaction trajectories (messages, actions, tool-call arguments/outputs) and assess safety using both an automated trajectory judge (AgentDoG-Qwen3-4B) and human review. Across 34 canonical cases, we find a non-uniform safety profile: performance is generally consistent on reliability-focused tasks, while most failures arise under underspecified intent, open-ended goals, or benign-seeming jailbreak prompts, where minor misinterpretations can escalate into higher-impact tool actions. We supplemented the overall results with representative case studies and summarized the commonalities of these cases, analyzing the security vulnerabilities and typical failure modes that Clawdbot is prone to trigger in practice.