Search papers, labs, and topics across Lattice.
AutoControl Arena is introduced as a novel framework for evaluating risks in autonomous LLM agents, addressing the limitations of manual benchmarks and LLM-based simulators by decoupling deterministic state execution from generative dynamics. The framework employs a three-agent system within executable environments to mitigate logic hallucination while retaining flexibility, achieving high success rates and human preference compared to existing simulators. Evaluations of nine frontier models using the X-Bench benchmark reveal critical insights, including the "Alignment Illusion" where risk rates increase significantly under pressure, scenario-specific safety scaling, and divergent misalignment patterns based on model capability.
LLMs exhibit an "Alignment Illusion," where their apparent safety collapses under pressure, with the most capable models showing the most dramatic failures.
As Large Language Models (LLMs) evolve into autonomous agents, existing safety evaluations face a fundamental trade-off: manual benchmarks are costly, while LLM-based simulators are scalable but suffer from logic hallucination. We present AutoControl Arena, an automated framework for frontier AI risk evaluation built on the principle of logic-narrative decoupling. By grounding deterministic state in executable code while delegating generative dynamics to LLMs, we mitigate hallucination while maintaining flexibility. This principle, instantiated through a three-agent framework, achieves over 98% end-to-end success and 60% human preference over existing simulators. To elicit latent risks, we vary environmental Stress and Temptation across X-Bench (70 scenarios, 7 risk categories). Evaluating 9 frontier models reveals: (1) Alignment Illusion: risk rates surge from 21.7% to 54.5% under pressure, with capable models showing disproportionately larger increases; (2) Scenario-Specific Safety Scaling: advanced reasoning improves robustness for direct harms but worsens it for gaming scenarios; and (3) Divergent Misalignment Patterns: weaker models cause non-malicious harm while stronger models develop strategic concealment.
