Search papers, labs, and topics across Lattice.
This paper introduces Dynamic Cyber Ranges, a novel evaluation environment for LLM-driven cybersecurity agents that incorporates LLM-driven Defender agents to provide real-time hardening and intrusion response. Experiments across three tiers of infrastructure show that Defender agents significantly reduce attacker success rates (0-55%), even achieving complete prevention in some configurations. The finding that a smaller, specialized on-premise model can match the defensive capabilities of frontier models, while also detecting attacks faster, highlights the potential for privacy-preserving, localized defense strategies.
Forget static defenses: LLM-powered "Defender" agents can dynamically harden cyber ranges, slashing attacker success rates and leveling the playing field as AI-driven threats evolve.
As LLM-driven agents advance in cybersecurity, Jeopardy CTF benchmarks are approaching saturation and cyber ranges, the natural next evaluation frontier, offer diminishing resistance under their current static design. We validate this observation by deploying an LLM-driven Advanced Persistent Threat (APT) agent across three tiers of increasingly realistic infrastructure (PRO Labs, MHBench, military-grade CYBER RANGES). To counteract this trend, we propose Dynamic Cyber Ranges: cyber range environments augmented with LLM-driven Defender agents that harden infrastructure, monitor for intrusions, and respond in real time. Across evaluated scenarios, Defender agents reduce attacker success to 0-55%, achieving complete prevention on multiple configurations. Since attacker and defender agents draw from the same underlying model capabilities, Dynamic Cyber Ranges preserve evaluation headroom as models improve. Notably, a smaller, specialized on-premise model (alias2-mini) matched the frontier model's defensive outcomes on multiple scenarios under identical, untuned prompts, and detected the attacker 10x faster on a complex enterprise scenario, suggesting that privacy-preserving on-premise models can serve as competent defenders against frontier-class attackers. The experiments further surface emergent agent behaviors, including scope expansion and prompt exfiltration, with implications for AI benchmark integrity and agentic system design.
