Demystify, Use, Reflect: Preparing students to be informed LLM-users

2025-11-13

AI Summary

Students lack structured, critical, and practice-oriented large language model (LLM) literacy. Method: We redesigned a post-CS1 course to systematically integrate four modules—LLM fundamentals, hands-on tool usage, ethics discussion, and reflective practice—and introduced a dual-strategy framework: “disclosure mechanisms” (requiring transparent AI use) and “validation training” (cultivating systematic verification of LLM outputs). Pedagogical approaches included LLM toolchain exercises, interactive classroom activities, pre-/post-intervention surveys, explicit critical thinking instruction, and an AI-augmented problem-solving framework. Contribution/Results: Students demonstrated deeper technical understanding, more deliberate and ethically grounded LLM usage, significantly enhanced validation awareness, and improved human-AI collaboration competence. The curriculum design is discipline-agnostic and offers a scalable, responsible pedagogical paradigm for computing education in an AI-integrated era.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-in-the-loop Machine Learning
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We transitioned our post-CS1 course that introduces various subfields of computer science so that it integrates Large Language Models (LLMs) in a structured, critical, and practical manner. It aims to help students develop the skills needed to engage meaningfully and responsibly with AI. The course now includes explicit instruction on how LLMs work, exposure to current tools, ethical issues, and activities that encourage student reflection on personal use of LLMs as well as the larger evolving landscape of AI-assisted programming. In class, we demonstrate the use and verification of LLM outputs, guide students in the use of LLMs as an ingredient in a larger problem-solving loop, and require students to disclose and acknowledge the nature and extent of LLM assistance. Throughout the course, we discuss risks and benefits of LLMs across CS subfields. In our first iteration of the course, we collected and analyzed data from students pre and post surveys. Student understanding of how LLMs work became more technical, and their verification and use of LLMs shifted to be more discerning and collaborative. These strategies can be used in other courses to prepare students for the AI-integrated future.

Problem

• Integrating LLMs into computer science education
• Developing responsible AI usage skills for students
• Addressing ethical considerations in AI-assisted programming

Innovation

• Integrates LLMs in structured critical practical manner
• Demonstrates use and verification of LLM outputs
• Guides students in LLM-assisted problem-solving loop