Validation of the Critical Reflection and Agency in Computing Index: Do Computing Ethics Courses Make a Difference?

2025-06-06

AI Summary

This study addresses the core question: “How can computational ethics education’s impact on students’ critical reflection and ethical agency be rigorously assessed?” Methodologically, it developed and validated the first domain-specific Critical Reflection and Agency Index (CRAI) for computing, grounded in two large-scale psychometric surveys (N = 474, 464). Rigorous measurement modeling established the first empirically supported multidimensional structure of ethical competence in computing. Results demonstrate excellent internal consistency across CRAI subscales (α > 0.82), strong construct validity, and cross-group measurement invariance. Ethics instruction significantly enhanced reflective depth and behavioral intentionality—but concurrently reinforced technological solutionism, exposing a fundamental tension between value-oriented reflection and technical centrism in prevailing pedagogical paradigms. The CRAI thus provides both a generalizable, evidence-based assessment instrument and novel theoretical insights to advance empirically grounded reform of computational ethics education.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Machine Learning: Ethics, Bias, and Fairness
• Application Domains: Humanities & Computational Social Science

Application Category

• Responsible Web: Fairness, accountability, transparency and ethics of web technologies
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Computing ethics education aims to develop students' critical reflection and agency. We need validated ways to measure whether our efforts succeed. Through two survey administrations (N=474, N=464) with computing students and professionals, we provide evidence for the validity of the Critical Reflection and Agency in Computing Index. Our psychometric analyses demonstrate distinct dimensions of ethical development and show strong reliability and construct validity. Participants who completed computing ethics courses showed higher scores in some dimensions of ethical reflection and agency, but they also exhibited stronger techno-solutionist beliefs, highlighting a challenge in current pedagogy. This validated instrument enables systematic measurement of how computing students develop critical consciousness, allowing educators to better understand how to prepare computing professionals to tackle ethical challenges in their work.

Problem

• Measure impact of computing ethics courses on students
• Validate Critical Reflection and Agency in Computing Index
• Assess ethical development and techno-solutionist beliefs

Innovation

• Validated Critical Reflection and Agency Index
• Psychometric analyses for ethical development
• Survey-based measurement of ethical consciousness