Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation

2026-04-18

AI Summary

This work addresses critical limitations in existing static benchmarks for harmful content detection—namely, their constrained scalability, limited diversity, and susceptibility to contamination from pretraining corpora. To overcome these issues, the authors propose a dynamic synthesis framework grounded in role-based simulation. By constructing two-dimensional user profiles that integrate demographic attributes with topical interests, the framework guides large language models to generate contextualized, highly harmful, and diverse content. The approach leverages role-guided agents, human-in-the-loop evaluation, and multidimensional analysis to produce data that significantly surpasses current benchmarks in terms of harmfulness, challenge level, and diversity. The resulting dataset exerts stronger stress on mainstream detection systems and achieves linguistic and thematic diversity comparable to manually curated datasets.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Humanities & Computational Social Science

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Static benchmarks for harmful content detection face limitations in scalability and diversity, and may also be affected by contamination from web-scale pre-training corpora. To address these issues, we propose a framework for synthesizing harmful content, leveraging persona-guided large language model (LLM) agents. Our approach constructs two-dimensional user personas by integrating demographic identities and topical interests with situational harmful strategies, enabling the simulation of diverse and contextually grounded harmful interactions. We evaluate the framework along three dimensions: harmfulness, challenge level, and diversity. Both human and LLM-based evaluations confirm that our framework achieves a high harmful generation success rate. Experiments across multiple detection systems reveal that our synthetic scenarios are more challenging to detect than those in existing benchmarks. Furthermore, a multi-faceted analysis confirms that our approach achieves linguistic and topical diversity comparable to human-curated datasets, establishing our framework as an effective tool for robust stress-testing of harmful content detection systems.

Problem

• harmful content detection
• static benchmarks
• scalability
• diversity
• benchmark contamination

Innovation

• persona-based simulation
• harmful content synthesis
• large language model agents
• robust evaluation
• contextual diversity