What LLMs Must Forget to Teach Effectively: A DIY Approach to Premodern Japanese Language Pedagogy

2026-05-31

AI Summary

This study addresses the pedagogical limitations of general-purpose large language models (LLMs) in educational settings, where excessive elaboration and hallucinations often undermine students’ active reasoning and diminish teacher–student agency in instruction. To counter this, the authors propose a “forgetting LLM” approach grounded in do-it-yourself (DIY) principles, leveraging prompt engineering to develop specialized tools tailored for classical Japanese language instruction. These include a text-analytic tutoring framework, a bilingual interactive dictionary, and a dialogic conversational partner—all designed to restore instructor and learner control over the AI-mediated teaching process. Implemented in graduate-level courses on classical Japanese literature and language, this bottom-up model of AI tool development demonstrates, through authentic dialogue data, its efficacy in fostering students’ proactive comprehension and alignment with curricular objectives.

Abstract

We discuss a novel approach to Premodern Japanese Language Pedagogy (PJLP) with potential applications in other languages and fields. The integration of artificial intelligence into education has largely operated as a top-down project, affording minimal agency to everyday users. This dynamic mirrors the broader frontier model ecosystem, which concentrates massive human and financial resources within a few labs. Drawing inspiration from grassroots initiatives such as the DIY and Maker movements, this paper advocates for an approach to AI in Education that fosters instructional and student agency over the pedagogical process. Specifically, we discuss a tutoring framework for textual analysis in the context of a graduate seminar in premodern Japanese literature, as well as a bilingual interactive dictionary and a conversational partner created for a language course in Classical Japanese. Created through prompt engineering as custom instances of a Large Language Model (LLM), these three tools are designed to counteract the tendency of out-of-the-box LLMs to either bypass student effort through over-explanation or misguide learners via hallucinations. To illustrate how this approach can promote active comprehension and pedagogical alignment, we provide transcripts (logs) of actual exchanges, sample instructions (system prompts), and guidance for instructors curious about exploring this approach in a variety of fields (starter kit).

Problem

• AI in Education
• Large Language Models
• Premodern Japanese Language Pedagogy
• Student Agency
• Hallucination

Innovation

• DIY AI
• prompt engineering
• Large Language Models
• pedagogical agency
• Classical Japanese