Measuring Curriculum Alignment across Topical Coverage, Competency, and Cognitive Depth: A Longitudinal Framework Applied to CS2013 and CS2023

2026-06-17

AI Summary

This study addresses the lack of reliable methods for evaluating how undergraduate computer science curricula align with international teaching guidelines such as CS2013 and CS2023, and how this alignment evolves across guideline revisions. The authors propose a human-in-the-loop analytical pipeline that structures course content and guideline knowledge units, employs semantic retrieval—incorporating reciprocal rank fusion and lightweight sentence embedding models—to generate matching candidates, and applies clearly defined coverage criteria for human validation, supplemented by Cohen’s kappa to assess inter-rater consistency. For the first time, the approach enables longitudinal measurement of curriculum alignment across three dimensions: topic coverage, competency expression, and cognitive depth, distinguishing structural gaps from changes due to updated standards. Empirical results reveal knowledge unit coverage rates of 50.9% for CS2013 and 49.7% for CS2023, competency coverage around 88%, but a notable decline in adherence to recommended cognitive depth—from 95% to 76%.

Abstract

Undergraduate computer science is governed by international curricular guidelines revised about once a decade, yet programs lack a reliable, reproducible way to measure how completely they cover the current guidelines and how that coverage shifts when the guidelines are restructured. We address this with a human-in-the-loop pipeline that measures a program's coverage of an external body of knowledge, applied longitudinally to one accredited BSc in Computer Science against Computer Science Curricula 2013 (CS2013) and 2023 (CS2023). The pipeline represents the program and each guideline as structured corpora, generates candidate course-to-knowledge-unit matches by semantic retrieval, and confirms them through human judgment under an explicit coverage definition. Of seven benchmarked retrievers, a reciprocal-rank-fusion ensemble was strongest, and a reputed long-context model underperformed a small sentence model, so retriever choice must be measured. Both maps were validated by an independent second rater (Cohen's kappa 0.64 for CS2023, 0.69 for CS2013). The program covers 49.7% of CS2023 and 50.9% of CS2013 knowledge units, near-constant across a decade. Extending the same retrieve-then-confirm design to competency articulation and cognitive depth shows that the program articulates the competency for ~88% of covered units under each guideline, yet delivers it at the recommended depth for 76% of present units under CS2023 against 95% under CS2013, a gap reflecting the newer guideline's raised expectations, not the program. The longitudinal comparison separates persistent structural gaps (parallel and distributed computing, foundations of programming languages, systems fundamentals), uncovered against both guidelines and ABET, from differences that reflect the standard's evolution. The instrument is reusable and available from the authors on request.

Problem

• curriculum alignment
• topical coverage
• competency
• cognitive depth
• longitudinal analysis

Innovation

• curriculum alignment
• semantic retrieval
• human-in-the-loop
• longitudinal analysis
• cognitive depth