Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators

2026-05-12

AI Summary

This study addresses a critical limitation in current large language model (LLM)-based student simulators: their inability to maintain consistent misconception-driven reasoning during interactive dialogue, leading to inauthentic learning behaviors. To tackle this, the authors introduce the first controllable evaluation framework that quantifies “misconception fidelity”—defined as the simulator’s capacity to revise its answers only when provided with feedback specifically targeting its core misconceptions—using a Misconception Contrastive Feedback protocol and a Selective Flip Score (SFS). The work reveals that existing LLMs exhibit strong sycophantic tendencies, evidenced by near-zero SFS values. To mitigate this, the authors propose a training paradigm combining supervised fine-tuning (SFT), preference optimization, and SFS-aligned reinforcement learning. Experiments demonstrate that SFT alone can improve SFS by up to 0.56, while SFS-aligned reinforcement learning yields more stable gains than preference optimization alone.

Abstract

Large language models (LLMs) can fluently generate student-like responses, making them attractive as simulated students for training and evaluating AI tutors and human educators. Yet such simulators are typically evaluated by output similarity to real students, not by whether they behave like students with coherent misconceptions during interaction. We introduce a controlled framework for evaluating misconception faithfulness, whether a simulator maintains a misconception-driven belief state and updates selectively when feedback addresses the underlying misconception. Central to our framework is a misconception-contrastive feedback protocol that compares targeted feedback against two controls: misaligned feedback (targeting a different but plausible misconception) and generic feedback (only identifying answer is wrong). We propose Selective Flip Score (SFS), which quantifies how much more often a simulator flips its answer under targeted feedback than under contrastive controls. Across seven LLMs (4B-120B), multiple datasets, and prompting strategies, simulators exhibit near-zero SFS, correcting their answers at similarly high rates regardless of feedback relevance. Further analyses reveal a sycophantic failure mode: models behave less like students with misconceptions but more like problem-solvers who treat any corrective signal as a cue to abandon the simulated belief and re-solve from internal knowledge. To address this, we develop a post-training pipeline spanning supervised fine-tuning (SFT), preference optimization, and reinforcement learning (RL) with an SFS-aligned reward; SFT yields notable gains up to +0.56, and SFS-aligned RL provides more consistent improvements than preference optimization. Our results establish misconception faithfulness as a challenging yet trainable property, motivating a shift from static output matching toward interactive, belief-aware student modeling.

Problem

• misconception faithfulness
• student simulation
• large language models
• interactive belief modeling
• sycophantic behavior

Innovation

• misconception faithfulness
• Selective Flip Score
• sycophantic behavior
• belief-aware simulation
• feedback-aligned RL