Search papers, labs, and topics across Lattice.
This paper introduces ActBench, a self-evolving benchmark designed to assess behavioral safety risks in cowork agents by evaluating execution trajectories instead of final outputs. The benchmark comprises 600 cases across 213 scenarios, targeting 15 distinct risk behaviors and leveraging a reward-guided beam search method to optimize both attack effectiveness and task utility. Evaluation of 15 large language models (LLMs) and 6 open-source cowork agents reveals significant variability in attack success rates, ranging from 10.1% to 94.4%, highlighting the critical need for dynamic safety assessments in AI systems.
Attack success rates in cowork agents can vary dramatically, with some models achieving up to 94.4% effectiveness in executing adversarial tasks.
Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API. We define behavioral safety and introduce ActBench, a self-evolving benchmark that evaluates such behavior risk from execution trajectories rather than final responses. Each case pairs a benign task with an adversarial variant that preserves its instruction, configuration, initial state, rating model, and trusted records while injecting a task-reachable payload. ActBench contains 600 cases from 213 scenarios, spanning 15 risk behaviors, six execution spaces, and 48 web-service APIs.To move beyond static payloads, we propose a reward-guided beam search method that jointly optimizes attack effectiveness and task utility, while reflection diagnoses failed execution checkpoint and guides payload revision. Besides, we propose a dual evidence verification mechanism that verifies agent execution safety and utility through log evidence and LLM-based trajectory evidence.We evaluate 15 LLMs and 6 open-source cowork agents over 24,000 trajectories. Under a fixed harness, attack success rates ranges from 10.1% to 94.4% across models, while under a fixed base model, they range from 73.7% to 94.4% across agents.These results show greater variation across models than agent harness, while attacks remain highly successful across all tested harnesses.Our benchmark is released at: https://github.com/zjuicsr/ActBench.
