Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents

2026-06-18

AI Summary

This work addresses the growing privacy and security risks posed by AI-powered autonomous web agents that commonly bypass protective mechanisms such as robots.txt. To counter this, the authors propose a multi-layer fingerprinting approach that integrates network-level features (e.g., TLS/HTTP protocol characteristics) with browser interaction behaviors, enabling precise differentiation among AI agents, human users, and traditional web crawlers. The method introduces a deployable logging framework that achieves fine-grained, cross-layer attribution of mainstream AI agents for the first time, demonstrating strong evasion resistance and robustness. Experimental evaluation across six widely used agent frameworks shows a classification accuracy of 97%, effectively isolating distinct traffic types and establishing a novel paradigm for content protection on the web.

Abstract

As AI web agents proliferate, combining large language models with autonomous, browser-level control, indiscriminate content scraping by web agents has emerged as a privacy and security challenge. Existing defenses, such as robots.txt and active bot-blocking, are insufficient, as they are widely violated and easily circumvented. In this work, we demonstrate that AI web agents can be effectively distinguished from humans and traditional crawlers using a multi-layer fingerprint based on both network layer characteristics (e.g., TLS, HTTP) and browser interaction behavior. We implement this mechanism as a programmatic logging framework that can be deployed on a live, instrumented domain. By analyzing six prominent agent frameworks (AutoGen, Browser Use, Claude, Gemini, Operator, and Skyvern), we uncover latent structural differences in how these systems assemble HTTP requests, establish TLS/HTTP connections, and execute autonomous browser actions. Feeding these multi-layer features into a decision tree classifier, our framework achieves high-fidelity identification (97% accuracy), successfully isolating distinct agent architectures and differentiating agent traffic from both human browsing baselines and legacy crawlers. Our findings demonstrate that cross-layer agent tracking provides a robust, evasion-resistant strategy for content protection and web security policy enforcement.

Problem

• AI web agents
• content scraping
• privacy
• web security
• bot detection

Innovation

• multi-layer fingerprinting
• autonomous web agents
• browser interaction behavior
• TLS/HTTP fingerprinting
• agent attribution