Detecting LLM-Assisted Academic Dishonesty using Keystroke Dynamics

2025-11-16

AI Summary

Generative AI poses novel academic integrity challenges, as conventional text-matching and intrinsic-feature detection methods struggle to identify AI-assisted writing and paraphrased plagiarism. Method: This paper proposes a behavior-level detection paradigm grounded in keystroke dynamics—modeling temporal input patterns (e.g., inter-keystroke intervals, edit frequency)—and integrates TypeNet, LightGBM, and CatBoost within a multimodal framework enhanced by adversarial training to improve robustness against synthetic inputs. Contribution/Results: We introduce a novel rewriting-aware module that significantly improves detection of LLM-generated and human-paraphrased text. Experiments demonstrate state-of-the-art performance: 97.2% F1-score in structured scenarios and 86.9% accuracy for paraphrased text detection using TypeNet—both substantially outperforming leading pure-text detectors and human evaluators.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The rapid adoption of generative AI tools has intensified the challenge of maintaining academic integrity. Conventional plagiarism detectors, which rely on text-matching or text-intrinsic features, often fail to identify submissions that have been AI-assisted or paraphrased. To address this limitation, we introduce keystroke-dynamics-based detectors that analyze how, rather than what, a person writes to distinguish genuine from assisted writing. Building on our earlier study, which collected keystroke data from 40 participants and trained a modified TypeNet model to detect assisted text, we expanded the dataset by adding 90 new participants and introducing a paraphrasing-based plagiarism-detection mode. We then benchmarked two additional gradient-boosting classifiers, LightGBM and CatBoost, alongside TypeNet, and compared their performance with DetectGPT, LLaMA 3.3 70B Instruct, and the results of 44 human evaluators. To further assess and improve robustness, we proposed a deception-based threat model simulating forged keystrokes and applied adversarial training as a countermeasure. Results show that the machine learning models achieve F1 scores above 97% in structured settings, while TypeNet performs best in detecting paraphrasing, with an F1 score of 86.9%. In contrast, text-only detectors and human evaluators perform near-chance, demonstrating that keystroke dynamics provide a strong behavioral signal for identifying AI-assisted plagiarism and support the use of multimodal behavioral features for reliable academic integrity assessment.

Problem

• Detecting AI-assisted academic dishonesty using keystroke dynamics analysis
• Overcoming limitations of conventional text-based plagiarism detection methods
• Identifying AI-assisted writing through behavioral patterns rather than content

Innovation

• Keystroke dynamics analyze writing behavior patterns
• Modified TypeNet model detects AI-assisted text generation
• Adversarial training counters forged keystroke deception threats