Towards Detecting AI-Assisted Responses in Online Surveys

2026-09-15

AI Summary

研究通过构建ASURRE数据集和使用多种LLM生成AI辅助调查响应，评估现有检测器性能，并提出一种简单的聚合方法以提高检测准确性。

Abstract

The use of LLMs to complete online surveys impacts the validity of survey-based research, but detecting such usage remains underexplored. We introduce an initial benchmark dataset, namely ASURRE, for AI-assisted survey participation to capture usage strategies ranging from full generation and revision to persona-grounded agentic completion. Controlled by these strategies, LLM-assisted survey responses are generated using multiple LLMs on three real-world surveys in different disciplines, paired with genuine human responses. Our evaluation of existing machine-generated text (MGT) detectors shows that naive AI usage is readily detectable, whereas persona-grounded agents that mimic entire respondents push detector performance toward chance. We further show that agentic completion cannot fully replicate respondent-level behaviour and leaves distinctive behavioural traces. While individual cues can be circumvented by targeted prompting, a simple few-shot, training-free aggregator over these cues improves mean AUROC by +0.14 over the best existing detector across agentic settings. Our project is available at https://github.com/mike-qz-wang/ASURRE.

Problem

• AI-assisted
• online surveys
• LLMs
• survey validity
• MGT detectors

Innovation

• ASURRE
• AI-assisted survey participation
• persona-grounded agents
• machine-generated text detectors
• few-shot aggregator