Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the detection of LLM-based Web Agents, which have emerged as sophisticated bots capable of bypassing traditional anti-bot mechanisms. By deploying honeysites protected by various anti-bot strategies and employing multi-layer fingerprinting techniques, the authors reveal that while some Web Agents can evade detection, all can be distinguished from human users and from each other. Notably, the study finds that stealth techniques often inadvertently enhance the detectability of these agents, challenging the assumption that such methods improve anonymity.
Stealth techniques for LLM-based Web Agents may paradoxically make them easier to detect, undermining their intended purpose.
Since 2023, a new class of bots has emerged: Web Agents. They can automate complex tasks on the Web, going beyond traditional browser automation tools such as Selenium, Puppeteer, or Playwright. Leveraging large language models (LLMs), these agents are capable of solving anti-bot mechanisms, mimicking human behavior, and, in some cases, operating directly from the local machine of the user configuring them. As a result, it is becoming increasingly difficult for website administrators to detect and block these LLM-based bots. Modern Web Agents commonly integrate stealth and anti-detection techniques, while numerous proprietary and open-source anti-bot mechanisms have emerged recently, specifically to block them. However, despite their growing prevalence, there is little evaluation of the effectiveness of state-of-the-art anti-bot mechanisms against these LLM-based bots and their stealth capabilities. Likewise, no prior work has comprehensively studied how to characterize and distinguish Web Agents deployed either in the cloud or locally. This paper addresses these open questions by deploying multiple honeysites protected by one or more anti-bot mechanisms (e.g., robots.txt, CAPTCHAs, proof-of-work, and Cloudflare's free proprietary solutions). We integrated network-, HTTP-, and browser-level fingerprinting techniques, and prompted six LLM-based Web Agents to visit the deployed honeysites. Our analysis reveals three main findings: (i) some Web Agents were able to bypass all evaluated anti-bot mechanisms; (ii) all evaluated Web Agents can be distinguished both from humans and from one another using multi-layer fingerprinting techniques across network, HTTP and browser layers; (iii) stealth and anti-detection mechanisms often increase detectability rather than decrease it.