The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions

2026-06-22

AI Summary

This work addresses the challenge of hidden misconceptions in student problem-solving, where learners often arrive at correct answers through flawed reasoning—rendering such errors invisible to conventional automated feedback systems. To tackle this issue, the authors propose a hierarchical assessment framework that decouples answer correctness from the validity of the solution process. They design a “detect–verify–escalate” pipeline that, under conditions of high uncertainty, triggers diagnostic follow-up questions rather than immediate alerts. Integrating a fine-tuned classifier with an open-source reasoning model and grounding the approach in educational theory, the system achieves low false-positive rates while maintaining high coverage of misconception detection. Empirical results demonstrate that the framework identifies 84% of hidden misconceptions, substantially reducing teacher intervention load and supporting dual deployment modes—teacher review and autonomous tutoring.

Abstract

Automated feedback systems that rely on answer correctness will reinforce, rather than address, misconceptions when students reach the correct answer through flawed reasoning. We investigate automatic detection of these hidden misconceptions using 20,964 real student responses from the Eedi mathematics platform. Fine-tuned classifiers detect only 57% of these hidden misconceptions, and standard ML interventions do not improve on this. An open-weight reasoning model detects 84%, but at realistic prevalence, false alarms outnumber genuine detections roughly 8 to 1. We present a graduated assessment rubric that separates answer correctness from method validity, and propose a detect-verify-escalate pipeline that routes uncertain cases to diagnostic follow-up questions rather than directly to teachers. Two deployment modes adapt the pipeline: a teacher dashboard where the system filters a review queue, and an autonomous tutor where flags trigger low-cost formative follow-up.

Problem

• hidden misconceptions
• automated feedback
• answer correctness
• flawed reasoning
• misconception detection

Innovation

• hidden misconceptions
• detect-verify-escalate pipeline
• graduated assessment rubric
• automated feedback
• reasoning validity