Investigating Youth's Technical and Ethical Understanding of Generative Language Models When Engaging in Construction and Deconstruction Activities

2025-04-21

AI Summary

This study addresses the challenge of fostering adolescents’ dual technical and ethical literacy regarding generative language models (GLMs). Targeting 14–15-year-old students, it implements a dual-track participatory workshop: (1) designing interpretable GLMs within a “glass-box” framework, and (2) systematically auditing commercial models’ behaviors through structured querying. Methodologically, it introduces the novel “in-pieces” analytical framework, integrating microgenetic knowledge analysis with ordered network analysis to dynamically trace the co-evolution of technical understanding and ethical reasoning. The study yields the first empirically grounded practice map of adolescent GLM design and audit activities. Results demonstrate that the complementary “construct–deconstruct” pedagogical approach significantly enhances the synergistic development of technical competence and ethical judgment—core dimensions of AI literacy. This work advances participatory AI education by bridging technical agency with critical ethical engagement in adolescent learners.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

The widespread adoption of generative artificial intelligence/machine learning (AI/ML) technologies has increased the need to support youth in developing AI/ML literacies. However, most work has centered on preparing young people to use these systems, with less attention to how they can participate in designing and evaluating them. This study investigates how engaging young people in the design and auditing of generative language models (GLMs) may foster the development of their understanding of how these systems work from both technical and ethical perspectives. The study takes an in-pieces approach to investigate novices' conceptions of GLMs. Such an approach supports the analysis of how technical and ethical conceptions evolve and relate to each other. I am currently conducting a series of participatory design workshops with sixteen ninth graders (ages 14-15) in which they will (a) build GLMs from a data-driven perspective that glassboxes how data shapes model performance and (b) audit commercial GLMs by repeatedly and systematically querying them to draw inferences about their behaviors. I will analyze participants' interactions to identify ethical and technical conceptions they may exhibit while designing and auditing GLMs. I will also conduct clinical interviews and use microgenetic knowledge analysis and ordered network analysis to investigate how participants' ethical and technical conceptions of GLMs relate to each other and change after the workshop. The study will contribute (a) evidence of how engaging youth in design and auditing activities may support the development of ethical and technical understanding of GLMs and (b) an inventory of novice design and auditing practices that may support youth's technical and ethical understanding of GLMs.

Problem

• Youth's technical and ethical understanding of generative language models.
• Engaging youth in designing and auditing AI systems for literacy development.
• Investigating evolution of technical and ethical conceptions in AI learning.

Innovation

• Engaging youth in GLM design and auditing
• Using data-driven GLM glassboxing techniques
• Analyzing ethical-technical conception evolution