TeachMateGPT: A Multi-Agent Knowledge-Grounded Framework for Pedagogical Assessment Generation from Science Curriculum Materials

2026-08-13

AI Summary

This study addresses the challenges of weak evidence grounding and poor syllabus alignment in low-resource course assessment generation by proposing a multi-agent question generation framework. Integrating graph retrieval with hybrid evidence matching, the approach employs COPE hierarchical indexing, a fail-closed pipeline, and the SAVER verification protocol to ensure content faithfulness and structural alignment. Additionally, this work introduces the NCTB-SciGen8 dataset. Experimental results demonstrate that, compared to baseline RAG methods, the proposed framework significantly improves faithfulness from 0.68 to 0.96 and answer relevance from 0.60 to 0.89. These findings confirm the method’s effectiveness in guaranteeing both the reliability and pedagogical applicability of scientifically generated examination questions.

Abstract

Automatically generating textbook-grounded assessment items can reduce science teachers' workload, but existing retrieval-augmented generation (RAG) systems rely on flat retrieval, support only single-question generation, lack safeguards against weak evidence, and are ill-suited to low-resource, board-exam-structured curricula. We address these limitations with TeachMateGPT, a multi-agent system contributing four advances to curriculum-grounded science-assessment authoring. (i) COPE, a hierarchical knowledge base replacing token-window chunking with a multi-resolution index that segments documents along syllabus structure and links them at three granularities via a traversable graph-based lineage, matching evidence to each topic's instructional level. (ii) A staged, fail-closed agent pipeline replacing one-shot retrieve-then-generate: routing gates search, retrieval fuses dense and lexical evidence under a coverage gate that withholds generation on insufficient evidence, and specialist agents draft objective and constructed-response items. (iii) SAVER, a source-attributed verification protocol scoring faithfulness, relevance, and hallucination risk against retrieved evidence, applying stricter grounding checks across each creative question's four sub-parts, paired with teacher-in-the-loop evaluation rather than automatic filtering. (iv) NCTB-SciGen8, a curriculum-grounded dataset of 198 items (143 multiple-choice, 55 creative questions) spanning all 14 chapters of the NCTB Class 8 science textbook, produced by the pipeline and rated by three practicing teachers. TeachMateGPT raises faithfulness (0.68 $\rightarrow$ 0.96) and answer relevancy (0.60 $\rightarrow$ 0.89) over a vanilla RAG baseline.

Problem

• Assessment Generation
• Retrieval-Augmented Generation
• Science Curriculum
• Low-Resource Settings
• Hallucination Mitigation

Innovation

• Multi-Agent System
• Hierarchical Knowledge Base
• Fail-Closed Pipeline
• Source-Attributed Verification
• Curriculum-Grounded Assessment