Black-Box Forensics for Conversational LLM Agents

2026-06-21

AI Summary

This work addresses the lack of provenance mechanisms for anonymous conversational large language model (LLM) agents that operate without access to model parameters or system prompts. The authors propose the first black-box fingerprinting method capable of identifying the underlying base model and determining whether distinct endpoints employ identical system prompts, using only a small number of ordinary dialogue interactions. Their approach integrates a base-model classifier with a cross-encoder architecture and leverages multi-turn dialogue aggregation to enhance discrimination performance. Experimental results demonstrate 98% accuracy in base-model attribution and an AUC of 0.768 (F1 = 0.703) for fingerprinting entirely unseen system prompts; aggregating responses across 50 dialogue turns significantly improves AUC to 0.943, highlighting strong generalization capabilities in open-world scenarios.

Abstract

As LLM-powered scams proliferate, black-box forensics for conversational LLM agents offers a path to accountability for systems hidden behind anonymous endpoints. Identifying the base model behind a chatbot endpoint (attribution), without model parameter access or knowledge of the hidden system prompt, would let investigators trace AI-enabled scams back to the providers whose models power them. Detecting when two endpoints run the exact same system prompt (fingerprinting), even one novel and unseen, would link individual scams into criminal networks and expose silent API changes. We conduct an empirical investigation of both capabilities. Our attribution classifiers identify the base model behind an agent with 98% accuracy from a few turns of non-adversarial conversation. Attribution of system prompts, while possible, requires retraining on a large amount of data for each prompt; system prompts in the wild are unbounded and ever-changing, making this approach costly. To tackle this more open-ended setting, our cross-encoder fingerprinting method achieves an AUC of 0.768 and an F1 of 0.703 on entirely unseen system prompts, and aggregating 50 interaction conversations from each target agent boosts AUC to 0.943. Conversational agents with unseen system prompts can thus be fingerprinted with robust accuracy from a few turns of ordinary conversation.

Problem

• black-box forensics
• conversational LLM agents
• attribution
• fingerprinting
• system prompt

Innovation

• black-box forensics
• attribution
• fingerprinting
• conversational LLM agents
• cross-encoder