Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation

2026-04-12

AI Summary

This work addresses critical limitations in current programming education approaches that rely on closed-source large language models (LLMs) to simulate student behavior, including privacy risks, high costs, and model dependency. The authors propose a novel method that first reformulates student programming process logs into alternating sequences of “code submissions” and “environmental feedback,” structured as dialogues. Leveraging this representation, they perform supervised fine-tuning and preference optimization on open-source LLMs such as Qwen-4B and Qwen-8B to better model authentic student debugging behaviors. Experimental results demonstrate that this approach significantly outperforms baseline methods—both code-only models and prompt-driven large models—in terms of functional alignment and code similarity, thereby substantially improving the fidelity of reconstructed student debugging trajectories.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Artificial models that simulate how learners act and respond within educational systems are a promising tool for evaluating tutoring strategies and feedback mechanisms at scale. However, many existing approaches in programming education rely on prompting large, proprietary language models, raising concerns around privacy, cost, and dependence. In this work, we propose a method for training open-weight artificial programming learners using authentic student process data. Our approach serializes temporal log traces into a conversational format, representing each student's problem-solving process as a dialogue between the learner and their automated assessment system. Student code submissions and environment feedback, such as test outcomes, grades, and error traces, form alternating conversational turns, enabling models to learn from the iterative debugging process. We additionally introduce a training pipeline combining supervised fine-tuning with preference optimization to align models with authentic student debugging behavior. We evaluate our framework by training Qwen models at 4B and 8B scales on a large-scale dataset of real student submissions to Python programming assignments. Our results show that incorporating environment feedback strengthens the models' ability to replicate student debugging behavior, improving over both prior code-only approaches and prompted large language models baselines in functional alignment and code similarity. We release our code to support reproducibility.

Problem

• student simulation
• programming education
• language models
• debugging behavior
• open-weight models

Innovation

• conversational serialization
• student simulation
• programming education
• preference optimization
• open-weight language models