Impact of AI Tools on Learning Outcomes: Decreasing Knowledge and Over-Reliance

2025-10-15

AI Summary

This study investigates the deep pedagogical impacts of generative AI tools on student learning. Motivated by concerns over diminished motivation, superficial knowledge acquisition, and cognitive substitution arising from students’ overreliance on AI for assignments and assessments in operations research courses, we conducted a randomized controlled trial: one group was permitted unrestricted AI use, while the other was prohibited from using AI throughout the course. Integrating grade compensation mechanisms, quantitative academic performance analysis, and educational-psychological behavioral observation, we provide the first empirical evidence—within a controlled instructional setting—that unfettered AI use significantly reduces classroom engagement, impairs conceptual mastery, and triggers systemic cognitive degradation. Students exhibit entrenched path dependence, undermining traditional assessment validity. Beyond establishing a causal link between AI misuse and declining learning quality, the study introduces the “cognitive substitution effect” as a novel theoretical framework, offering critical empirical foundations for rethinking educational interventions and assessment design in the age of artificial intelligence.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Philosophy and Ethics of AI: Artificial General Intelligence

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Students at all levels of education are increasingly relying on generative artificial intelligence (AI) tools to complete assignments and achieve higher exam scores. However, it remains unclear how this reliance affects their motivation, their genuine understanding of the material, and the extent to which it substitutes for the process of knowledge acquisition. To investigate the impact of generative AI on learning outcomes, an experiment was conducted at Corvinus University of Budapest. In an operations research class, students were randomly assigned into two groups: one was permitted to use AI tools during classes and examinations, while the other was not. To ensure fairness, a compensation mechanism was introduced: students in the lower-performing group received point adjustments until the average performance of the two groups was equalized. Despite the organizers' best efforts to explain the design and to create equal opportunities for all participants, many students perceived the experiment as a major disruption. Although the experiment was approved by every relevant university authority -- including the Ethics Board, the Head of Department, the Program Director, and the Student Council -- students escalated their concerns to the media and eventually to the State Secretary for Higher Education of Hungary. As a result, the experiment had to be substantially revised before completion: on the final exam the test group was merged with the control group. Still, the data allowed us to draw decisive conclusions regarding the students' learning habits. Uncontrolled use of AI tools leads to disengaged students and low understanding of material. The extreme reactions of the students proved even more revealing than the data collected: generative AI tools have already become indispensable for students, raising fundamental questions about the validity of their learning process.

Problem

• Investigating AI tools' impact on student learning outcomes and knowledge retention
• Examining how AI reliance affects motivation and genuine understanding of material
• Assessing whether AI tools substitute for authentic knowledge acquisition processes

Innovation

• Randomized controlled trial comparing AI and non-AI groups
• Compensation mechanism equalized average group performance
• Final exam merged test and control groups