Detecting LLM-assisted writing in scientific communication: Are we there yet?

2024-01-30

Journal of Data and Information Science

AI Summary

This study addresses the lack of transparency in LLM-assisted scientific writing by systematically evaluating four state-of-the-art LLM-generated text detectors under realistic research scenarios. We find that general-purpose detectors perform significantly worse than a simple time-based heuristic—namely, detecting abrupt stylistic shifts—when identifying hybrid scientific texts (human-authored + LLM-generated). Our work is the first to reveal the severe inadequacy of current detection methods in handling gradual, non-uniform LLM integration patterns prevalent in scientific writing. Consequently, we argue for the development of a domain-specific detection framework tailored to the mixed-authorship characteristics of scientific texts. A key contribution is the empirical validation that stylistic discontinuity—not just lexical or statistical anomalies—serves as a more robust detection signal. This insight establishes a new paradigm for ensuring traceability and upholding academic integrity in LLM-augmented scholarly communication.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Abstract Large Language Models (LLMs), exemplified by ChatGPT, have significantly reshaped text generation, particularly in the realm of writing assistance. While ethical considerations underscore the importance of transparently acknowledging LLM use, especially in scientific communication, genuine acknowledgment remains infrequent. A potential avenue to encourage accurate acknowledging of LLM-assisted writing involves employing automated detectors. Our evaluation of four cutting-edge LLM-generated text detectors reveals their suboptimal performance compared to a simple ad-hoc detector designed to identify abrupt writing style changes around the time of LLM proliferation. We contend that the development of specialized detectors exclusively dedicated to LLM-assisted writing detection is necessary. Such detectors could play a crucial role in fostering more authentic recognition of LLM involvement in scientific communication, addressing the current challenges in acknowledgment practices.

Problem

• Detecting LLM-assisted writing in scientific communication
• Evaluating performance of existing AI-generated text detectors
• Developing specialized detectors for accurate LLM use acknowledgment

Innovation

• Evaluated four cutting-edge LLM-generated text detectors
• Proposed simple ad-hoc detector for writing style changes
• Advocated developing specialized detectors for LLM-assisted writing