Reviving Reflection-in-Action: Instilling Designerly Thinking in AI-Supported Ideation through Multimodal Prompting

2026-06-25

AI Summary

This study addresses the limitations of current AI-based creative support tools, which overly rely on text prompts and may hinder designers’ reflective engagement and iterative thinking—potentially undermining students’ core design competencies. To counter this, the authors propose SketchifAI, a novel prototype system that introduces multimodal input (text, sketch, and sketch-with-labels) into AI-augmented design education. By leveraging sketching to create “productive friction,” SketchifAI reinvigorates reflective-in-action design thinking. Through a within-subject mixed-methods study integrating multimodal interaction analysis, divergent thinking assessments, and interviews, the research demonstrates that sketch-based modalities significantly enhance ideational fluency, even though users still express a preference for text. These findings offer a promising pathway toward AI tools that balance generative efficiency with the cultivation of human-centered design cognition, supporting rather than supplanting essential designerly skills.

Abstract

Current AI-powered creativity support tools (AI-CSTs) primarily use text prompting to generate solution-oriented outputs. However, the potential value of multimodal prompting in designer-AI interaction, specifically the introduction of productive friction to encourage iteration and reflection, has not been fully explored. To address this, we developed SketchifAI, a prototype AI-CST, and evaluated it with design students. In a mixed-methods, within-participants study, we examined how different input modalities (text, sketch, and sketch-plus-tags) affected design students' perceived ability to express their intent, their perception of creativity support, and their divergent thinking performance. Our preliminary findings suggest that the sketch modality tended to enhance fluency, with inconclusive evidence for differences in variety, originality, or quality compared to text modality. Yet, paradoxically, participants showed a strong preference for text prompting. We discuss how AI tools might be designed to reintroduce reflection-through-sketching, ensuring that designer-AI interaction supports, rather than erodes, essential design skills in students.

Problem

• AI-powered creativity support tools
• multimodal prompting
• reflection-in-action
• designer-AI interaction
• design education

Innovation

• multimodal prompting
• reflection-in-action
• AI creativity support tools
• designerly thinking
• sketch-based interaction