Knowledge is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis

2025-02-08

AI Summary

Cognitive diagnosis faces challenges in modeling sparse student-item interactions, while existing approaches suffer from two key limitations: (i) large language models (LLMs) lack fine-grained behavioral modeling capabilities, and (ii) traditional cognitive diagnosis models (CDMs) operate in behaviorally grounded but semantically impoverished representation spaces. To bridge this semantic–behavioral gap, we propose Knowledge-enhanced Cognitive Diagnosis (KCD), a model-agnostic two-stage framework. First, an LLM performs fine-grained joint semantic encoding of students and items. Second, contrastive learning and masked reconstruction align the LLM’s semantic space with the CDM’s behavioral space. KCD significantly improves diagnostic accuracy—especially for low-frequency students and items—demonstrating enhanced generalization. Extensive experiments on multiple real-world educational datasets validate that knowledge injection effectively bridges the representational misalignment between semantic and behavioral modeling.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Data Mining & Knowledge Management: Semantic Web

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Cognitive Diagnosis Models (CDMs) are designed to assess students' cognitive states by analyzing their performance across a series of exercises. However, existing CDMs often struggle with diagnosing infrequent students and exercises due to a lack of rich prior knowledge. With the advancement in large language models (LLMs), which possess extensive domain knowledge, their integration into cognitive diagnosis presents a promising opportunity. Despite this potential, integrating LLMs with CDMs poses significant challenges. LLMs are not well-suited for capturing the fine-grained collaborative interactions between students and exercises, and the disparity between the semantic space of LLMs and the behavioral space of CDMs hinders effective integration. To address these issues, we propose a novel Knowledge-enhanced Cognitive Diagnosis (KCD) framework, which is a model-agnostic framework utilizing LLMs to enhance CDMs and compatible with various CDM architectures. The KCD framework operates in two stages: LLM Diagnosis and Cognitive Level Alignment. In the LLM Diagnosis stage, both students and exercises are diagnosed to achieve comprehensive and detailed modeling. In the Cognitive Level Alignment stage, we bridge the gap between the CDMs' behavioral space and the LLMs' semantic space using contrastive learning and mask-reconstruction approaches. Experiments on several real-world datasets demonstrate the effectiveness of our proposed framework.

Problem

• Enhancing Cognitive Diagnosis Models with LLMs
• Bridging semantic and behavioral spaces
• Improving diagnosis for infrequent students and exercises

Innovation

• Integrates LLMs with CDMs
• Uses contrastive learning techniques
• Implements Cognitive Level Alignment stage