Investigating the Impact of Dark Patterns on LLM-Based Web Agents

2025-10-20

AI Summary

This study addresses the previously unexplored impact of dark patterns—deceptive UI designs—on decision-making by large language model (LLM)-based general-purpose web agents. Method: We introduce LiteAgent, a lightweight agent framework, and TrickyArena, a controllable, multi-scenario benchmark platform featuring real-world websites (e.g., e-commerce, streaming) embedded with dark patterns. Using automated task execution, interaction logging, and screen recording analysis, we evaluate six state-of-the-art LLM agents across diverse dark pattern categories. Results: Visual and HTML-level manipulations—and their multimodal combinations—significantly impair agent reasoning; under single dark patterns, agents fail 41% of tasks on average, with susceptibility strongly dependent on interface modification type. Contribution: We uncover a novel vulnerability of LLM agents in realistic web environments and propose the first configurable, reproducible evaluation framework for assessing dark pattern robustness—establishing an empirical foundation and standardized benchmark for developing trustworthy AI web agents.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Intelligent User Interfaces

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

As users increasingly turn to large language model (LLM) based web agents to automate online tasks, agents may encounter dark patterns: deceptive user interface designs that manipulate users into making unintended decisions. Although dark patterns primarily target human users, their potentially harmful impacts on LLM-based generalist web agents remain unexplored. In this paper, we present the first study that investigates the impact of dark patterns on the decision-making process of LLM-based generalist web agents. To achieve this, we introduce LiteAgent, a lightweight framework that automatically prompts agents to execute tasks while capturing comprehensive logs and screen-recordings of their interactions. We also present TrickyArena, a controlled environment comprising web applications from domains such as e-commerce, streaming services, and news platforms, each containing diverse and realistic dark patterns that can be selectively enabled or disabled. Using LiteAgent and TrickyArena, we conduct multiple experiments to assess the impact of both individual and combined dark patterns on web agent behavior. We evaluate six popular LLM-based generalist web agents across three LLMs and discover that when there is a single dark pattern present, agents are susceptible to it an average of 41% of the time. We also find that modifying dark pattern UI attributes through visual design changes or HTML code adjustments and introducing multiple dark patterns simultaneously can influence agent susceptibility. This study emphasizes the need for holistic defense mechanisms in web agents, encompassing both agent-specific protections and broader web safety measures.

Problem

• Investigating dark patterns' impact on LLM-based web agents' decision-making
• Assessing agent susceptibility to deceptive UI designs in web interactions
• Developing frameworks to evaluate dark pattern effects on automated agents

Innovation

• LiteAgent framework automates agent task execution
• TrickyArena environment controls dark pattern testing
• Visual and HTML modifications assess agent susceptibility