On the Suitability of LLM-Driven Agents for Dark Pattern Audits

Chen Sun; Yash Vekaria; Rishab Nithyanand

doi:10.48550/arxiv.2603.03881

Back

On the Suitability of LLM-Driven Agents for Dark Pattern Audits

Preprint

Open access

On the Suitability of LLM-Driven Agents for Dark Pattern Audits

Chen Sun, Yash Vekaria and Rishab Nithyanand

ArXiv.org

Cornell University

03/04/2026

DOI: 10.48550/arxiv.2603.03881

Files and links (1)

url

https://doi.org/10.48550/arxiv.2603.03881View

Preprint (Author's original)This preprint has not been evaluated by subject experts through peer review. Preprints may undergo extensive changes and/or become peer-reviewed journal articles. Open Access

Abstract

As LLM-driven agents begin to autonomously navigate the web, their ability to interpret and respond to manipulative interface design becomes critical. A fundamental question that emerges is: can such agents reliably recognize patterns of friction, misdirection, and coercion in interface design (i.e., dark patterns)? We study this question in a setting where the workflows are consequential: website portals associated with the submission of CCPA-related data rights requests. These portals operationalize statutory rights, but they are implemented as interactive interfaces whose design can be structured to facilitate, burden, or subtly discourage the exercise of those rights. We design and deploy an LLM-driven auditing agent capable of end-to-end traversal of rights-request workflows, structured evidence gathering, and classification of potential dark patterns. Across a set of 456 data broker websites, we evaluate: (1) the ability of the agent to consistently locate and complete request flows, (2) the reliability and reproducibility of its dark pattern classifications, and (3) the conditions under which it fails or produces poor judgments. Our findings characterize both the feasibility and the limitations of using LLM-driven agents for scalable dark pattern auditing.

Computer Science - Artificial Intelligence

Computer Science - Computation and Language

Computer Science - Computers and Society

Computer Science - Cryptography and Security

Computer Science - Human-Computer Interaction

Details

Title: Subtitle: On the Suitability of LLM-Driven Agents for Dark Pattern Audits
Creators: Chen Sun - University of Iowa
Yash Vekaria - University of California, Davis
Rishab Nithyanand - University of Iowa
Resource Type: Preprint
Publication Details: ArXiv.org
DOI: 10.48550/arxiv.2603.03881
ISSN: 2331-8422
Publisher: Cornell University; Ithaca, New York
Language: English
Date posted: 03/04/2026
Academic Unit: Center for Social Science Innovation; Computer Science; Law Faculty
Record Identifier: 9985141874102771

Metrics

4 Record Views