LLM4CD: Leveraging Large Language Models for Open-World Knowledge Augmented Cognitive Diagnosis

2025-05-14

AI Summary

Existing cognitive diagnosis models rely on ID-based embeddings, limiting their ability to capture semantic relationships inherent in educational data and rendering them ineffective for cold-start scenarios involving new students or items, as well as open-world adaptation. To address these limitations, we propose a semantic-enhanced cognitive diagnosis framework featuring a novel two-level encoder architecture: a macro-level cognitive text encoder that captures holistic student learning patterns, and a micro-level knowledge state encoder that models fine-grained concept mastery. Crucially, we replace conventional ID embeddings with large language model–driven, knowledge-aware textual representations, enabling zero-shot generalization. Extensive experiments across multiple real-world educational datasets demonstrate significant improvements in diagnostic accuracy and cross-scenario generalizability. Our results validate that semantic modeling effectively synergistically enhances both precision and openness in cognitive diagnosis.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Cognitive diagnosis (CD) plays a crucial role in intelligent education, evaluating students' comprehension of knowledge concepts based on their test histories. However, current CD methods often model students, exercises, and knowledge concepts solely on their ID relationships, neglecting the abundant semantic relationships present within educational data space. Furthermore, contemporary intelligent tutoring systems (ITS) frequently involve the addition of new students and exercises, a situation that ID-based methods find challenging to manage effectively. The advent of large language models (LLMs) offers the potential for overcoming this challenge with open-world knowledge. In this paper, we propose LLM4CD, which Leverages Large Language Models for Open-World Knowledge Augmented Cognitive Diagnosis. Our method utilizes the open-world knowledge of LLMs to construct cognitively expressive textual representations, which are then encoded to introduce rich semantic information into the CD task. Additionally, we propose an innovative bi-level encoder framework that models students' test histories through two levels of encoders: a macro-level cognitive text encoder and a micro-level knowledge state encoder. This approach substitutes traditional ID embeddings with semantic representations, enabling the model to accommodate new students and exercises with open-world knowledge and address the cold-start problem. Extensive experimental results demonstrate that our proposed method consistently outperforms previous CD models on multiple real-world datasets, validating the effectiveness of leveraging LLMs to introduce rich semantic information into the CD task.

Problem

• Enhancing cognitive diagnosis with semantic relationships using LLMs
• Addressing cold-start issues for new students and exercises
• Improving CD accuracy via open-world knowledge integration

Innovation

• Leverages LLMs for open-world knowledge augmentation
• Uses bi-level encoder for cognitive text encoding
• Replaces ID embeddings with semantic representations