Search papers, labs, and topics across Lattice.
This paper introduces a fully automated framework for adversary emulation that integrates playbook generation, execution, and failure recovery using LLMs, addressing limitations in prior approaches that required manual intervention. The framework generates Caldera playbooks from MITRE ATT&CK-aligned cyber threat intelligence (CTI) reports and employs a failure-type-aware recovery mechanism to enhance execution success rates. Evaluations on multiple LLMs demonstrated significant improvements in execution success and overall effectiveness, with the best performance achieved using Claude Sonnet 4.5, which surpassed the state-of-the-art AURORA system.
Automated adversary emulation can achieve over 84% execution success by intelligently revising playbooks based on real-time failures.
This paper presents a fully automated end-to-end framework for adversary emulation from MITRE ATT&CK-aligned CTI reports using LLMs. Unlike prior work, which either executes prewritten playbooks or partially automates playbook generation, our framework unifies playbook generation, execution, and failure recovery in a single workflow. In particular, although AURORA, the most recent prior study, generates playbooks from CTI reports, it still requires partial manual intervention and does not revise playbooks based on execution failures. Our framework generates Caldera playbooks from CTI reports, executes them automatically, and revises failed Abilities through a failure-type-aware recovery mechanism. Evaluated on 11 CTI reports with Claude Sonnet 4.5, GPT-4o, Gemini 2.5 Pro, and Grok 4 Fast, the framework achieved its best results with Claude Sonnet 4.5: 27.3 Abilities per playbook, 84.22% execution success after revision, and CTI Precision, Recall, and F1 of 73.95%, 52.48%, and 60.50%, respectively. The failure recovery mechanism consistently improved execution success across all evaluated LLM models by 14.59%p to 17.23%p. On the 10 CTI reports selected from AURORA's dataset, this mechanism further increased the final execution success rate, surpassing that of AURORA, which represents the state-of-the-art adversary emulation system.
