Search papers, labs, and topics across Lattice.
This study systematically evaluates the effectiveness of web bot defenses against LLM-based browser agents and commercial Captcha-solving services. The authors find that traditional challenge-based defenses are largely ineffective, with commercial solvers achieving near-perfect bypass rates, while LLM agents can also circumvent these defenses when equipped with dedicated solver modules. Notably, the resilience of non-interactive defenses like reCaptcha v3 is shown to be superficial, hinging on execution-environment authenticity rather than agent behavior, which has critical implications for the design of bot management systems.
Traditional web bot defenses crumble under the pressure of LLM agents and commercial solvers, revealing a critical vulnerability in current security architectures.
LLM-based browser agents are rapidly changing the threat landscape for web security. Unlike traditional automation frameworks that execute predefined scripts, these agents can autonomously navigate websites, reason about page content, and interact with web interfaces using natural-language instructions. This evolution raises fundamental questions about the effectiveness of bot management systems, widely deployed to defend against automated web abuse. In this paper, we present a systematic measurement study evaluating the resilience of both interactive challenge-based defenses and non-interactive trust-based defenses against two attacker classes: commercial Captcha-solving services and LLM-based browser agents. Our evaluation spans seven solver services and six agents, including cloud-hosted, self-hosted, AI-assisted, and browser-extension configurations, tested against hCaptcha, reCaptcha v2, reCaptcha v3, and Cloudflare Turnstile. Our results show that challenge-based defenses are broadly ineffective against commercial solvers, which achieve near-perfect bypass at negligible cost. The challenges can similarly be defeated by LLM-based agents when a dedicated solver module is available. Non-interactive defenses such as reCaptcha v3 exhibit stronger resistance, but our analysis reveals that this resilience does not reflect a fundamental security property. Through fine-grained interaction trace analysis, we find that two agents with nearly indistinguishable behavioral footprints yield divergent outcomes, one bypassing the defense and one failing, isolating execution-environment authenticity, rather than agent behavior, as the determining factor. These findings suggest that the security boundary of non-interactive defenses lies at the environment layer, with significant implications for how bot management systems are designed and evaluated.
