When and How AI Should Assist Brainstorming for AI Impact Assessment

2026-04-30

AI Summary

This study addresses the limitations of existing AI tools in facilitating collaborative impact assessments by diverse teams, which often lack designs and evaluations tailored to collaborative brainstorming contexts and struggle to effectively integrate pluralistic perspectives. Drawing on strategic foresight methodologies, the authors propose a phased human-AI collaboration strategy informed by co-design and field workshop experiments: during early ideation, the AI provides prompts rather than solutions; in the convergence phase, it supports structured synthesis; and throughout, it focuses on streamlining cumbersome processes rather than replacing human creativity. Mixed-methods evaluation demonstrates that this approach significantly enhances both the quality of impact assessments and collaborative experience in general-purpose AI scenarios (e.g., chatbots), though its efficacy is limited in highly specialized domains such as kidney allocation.

Abstract

A key task in AI practice is to assess potential impacts to prevent harm. Current AI tools assisting AI impact assessment have not been designed or evaluated for collaborative team brainstorming, and they do not capture the range of views in diverse teams. We studied how AI can support team brainstorming during AI impact assessment and made three contributions. First, we adapted two structured methods from strategic foresight and co-designed AI interventions for them in five in-person workshops with 28 participants in total. Second, we evaluated the interventions in ten in-person workshops with 54 participants, finding that AI improved impact assessment quality and brainstorming perceptions for a general-purpose AI use (a chatbot companion) but not for a specialised one (a kidney allocation application). Third, our findings result in broader design guidance for AI assistance in brainstorming: AI should only offer hints and not solutions during early ideation, initiating interaction only when participants face fixation or saturation; it should facilitate structuring ideas during convergence; leverage expertise to refine ideas; and overall, it should serve more in support of tedious brainstorming process tasks, rather than ideation that teams value to do themselves.

Problem

• AI impact assessment
• team brainstorming
• collaborative ideation
• diverse teams
• AI assistance

Innovation

• AI-assisted brainstorming
• AI impact assessment
• collaborative ideation
• design guidelines
• strategic foresight