Search papers, labs, and topics across Lattice.
This paper investigates the suitability of LLM-driven agents for auditing dark patterns in CCPA data rights request portals. An LLM agent was designed to autonomously navigate websites, complete request flows, gather evidence, and classify dark patterns across 456 data broker websites. The study evaluates the agent's consistency, reliability, and failure modes in identifying manipulative interface designs. The findings reveal both the potential and limitations of using LLMs for scalable dark pattern auditing.
LLM agents can audit dark patterns in data privacy request workflows, but their reliability and reproducibility vary significantly across websites.
As LLM-driven agents begin to autonomously navigate the web, their ability to interpret and respond to manipulative interface design becomes critical. A fundamental question that emerges is: can such agents reliably recognize patterns of friction, misdirection, and coercion in interface design (i.e., dark patterns)? We study this question in a setting where the workflows are consequential: website portals associated with the submission of CCPA-related data rights requests. These portals operationalize statutory rights, but they are implemented as interactive interfaces whose design can be structured to facilitate, burden, or subtly discourage the exercise of those rights. We design and deploy an LLM-driven auditing agent capable of end-to-end traversal of rights-request workflows, structured evidence gathering, and classification of potential dark patterns. Across a set of 456 data broker websites, we evaluate: (1) the ability of the agent to consistently locate and complete request flows, (2) the reliability and reproducibility of its dark pattern classifications, and (3) the conditions under which it fails or produces poor judgments. Our findings characterize both the feasibility and the limitations of using LLM-driven agents for scalable dark pattern auditing.
