Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks

2026-04-20

AI Summary

This work addresses a critical vulnerability in large language model (LLM)-based educational assistants: their tendency to inadvertently disclose complete answers when interacting with adversarial students, thereby violating core pedagogical principles. For the first time, adversarial attacks are introduced into educational settings through the development and fine-tuning of learnable adversarial student agents that emulate realistic attack behaviors. The study systematically evaluates the robustness of diverse LLM teaching assistants under six education-specific attack strategies. Results reveal that current assistants are generally susceptible to such attacks, and the proposed adversarial agents effectively induce answer leakage. Notably, even simple defense mechanisms substantially enhance assistant robustness. This research establishes the first standardized benchmark for evaluating the security of educational AI systems and offers practical defensive strategies to mitigate adversarial exploitation.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) are increasingly used in education, yet their default helpfulness often conflicts with pedagogical principles. Prior work evaluates pedagogical quality via answer leakage-the disclosure of complete solutions instead of scaffolding-but typically assumes well-intentioned learners, leaving tutor robustness under student misuse largely unexplored. In this paper, we study scenarios where students behave adversarially and aim to obtain the correct answer from the tutor. We evaluate a broad set of LLM-based tutor models, including different model families, pedagogically aligned models, and a multi-agent design, under a range of adversarial student attacks. We adapt six groups of adversarial and persuasive techniques to the educational setting and use them to probe how likely a tutor is to reveal the final answer. We evaluate answer leakage robustness using different types of in-context adversarial student agents, finding that they often fail to carry out effective attacks. We therefore introduce an adversarial student agent that we fine-tune to jailbreak LLM-based tutors, which we propose as the core of a standardized benchmark for evaluating tutor robustness. Finally, we present simple but effective defense strategies that reduce answer leakage and strengthen the robustness of LLM-based tutors in adversarial scenarios.

Problem

• answer leakage
• LLM tutors
• adversarial student attacks
• robustness
• educational AI

Innovation

• answer leakage
• adversarial student attacks
• LLM robustness
• jailbreak agent
• pedagogical alignment