AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages

2025-05-15

AI Summary

In early industrial AI design, cross-functional teams struggle to effectively communicate technical concepts to non-technical stakeholders, resulting in insufficient identification of ethical risks. To address this, we propose a “modular” collaborative framework for Responsible AI (RAI) in early design stages, integrating phased checklists, LLM-driven persona simulation, and interactive visualization modules—enabling non-technical stakeholders to understand and actively participate in systematic harm assessment. Implemented as a web-based prototype, the framework improves harm identification count by 42% and detection probability by 38%. User feedback highlights enhanced accessibility and collaboration quality, attributed to its clear structural organization and well-designed role-specific prompts. This work presents the first integration of a modular toolchain with LLM-enabled, cross-role collaboration mechanisms, offering a scalable, low-barrier engineering approach to operationalizing RAI in early development.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: Human-AI Collaboration / Human-AI Teaming
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

Responsible AI (RAI) efforts increasingly emphasize the importance of addressing potential harms early in the AI development lifecycle through social-technical lenses. However, in cross-functional industry teams, this work is often stalled by a persistent knowledge handoff challenge: the difficulty of transferring high-level, early-stage technical design rationales from technical experts to non-technical or user-facing roles for ethical evaluation and harm identification. Through literature review and a co-design study with 8 practitioners, we unpack how this challenge manifests -- technical design choices are rarely handed off in ways that support meaningful engagement by non-technical roles; collaborative workflows lack shared, visual structures to support mutual understanding; and non-technical practitioners are left without scaffolds for systematic harm evaluation. Existing tools like JIRA or Google Docs, while useful for product tracking, are ill-suited for supporting joint harm identification across roles, often requiring significant extra effort to align understanding. To address this, we developed AI LEGO, a web-based prototype that supports cross-functional AI practitioners in effectively facilitating knowledge handoff and identifying harmful design choices in the early design stages. Technical roles use interactive blocks to draft development plans, while non-technical roles engage with those blocks through stage-specific checklists and LLM-driven persona simulations to surface potential harms. In a study with 18 cross-functional practitioners, AI LEGO increased the volume and likelihood of harms identified compared to baseline worksheets. Participants found that its modular structure and persona prompts made harm identification more accessible, fostering clearer and more collaborative RAI practices in early design.

Problem

• Addressing knowledge handoff challenges in cross-functional AI teams
• Lack of shared visual structures for mutual understanding in RAI
• Inadequate tools for systematic harm evaluation in early AI design

Innovation

• Web-based prototype for cross-functional AI collaboration
• Interactive blocks for technical design planning
• LLM-driven persona simulations for harm identification