Feed-O-Meter: Fostering Design Feedback Skills through Role-playing Interactions with AI Mentee

2025-09-09

AI Summary

In design education, students’ underdeveloped feedback literacy is often impeded by low self-efficacy and fear of judgment. To address this, we propose an AI-augmented feedback training system grounded in role reversal: learners assume the mentor role and deliver design critiques to an anthropomorphized large language model (LLM) acting as an “AI apprentice,” then reflect on how their feedback dynamically shapes the AI’s iterative design evolution via an interactive visualization module. Our key contribution lies in engineering an LLM agent with a responsive feedback mechanism—enabled by contextual prompt engineering and multi-turn pedagogical dialogue—thereby overcoming the superficiality typical of conventional role-play approaches. A user study (N=24) demonstrates statistically significant improvements in learner engagement, clarity of feedback articulation, and contextual adaptability, validating the system’s efficacy and feasibility for cultivating critical feedback competencies in design education.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Effective feedback, including critique and evaluation, helps designers develop design concepts and refine their ideas, supporting informed decision-making throughout the iterative design process. However, in studio-based design courses, students often struggle to provide feedback due to a lack of confidence and fear of being judged, which limits their ability to develop essential feedback-giving skills. Recent advances in large language models (LLMs) suggest that role-playing with AI agents can let learners engage in multi-turn feedback without the anxiety of external judgment or the time constraints of real-world settings. Yet prior studies have raised concerns that LLMs struggle to behave like real people in role-play scenarios, diminishing the educational benefits of these interactions. Therefore, designing AI-based agents that effectively support learners in practicing and developing intellectual reasoning skills requires more than merely assigning the target persona's personality and role to the agent. By addressing these issues, we present Feed-O-Meter, a novel system that employs carefully designed LLM-based agents to create an environment in which students can practice giving design feedback. The system enables users to role-play as mentors, providing feedback to an AI mentee and allowing them to reflect on how that feedback impacts the AI mentee's idea development process. A user study (N=24) indicated that Feed-O-Meter increased participants' engagement and motivation through role-switching and helped them adjust feedback to be more comprehensible for an AI mentee. Based on these findings, we discuss future directions for designing systems to foster feedback skills in design education.

Problem

• Developing AI role-playing systems to teach design feedback skills
• Addressing student anxiety and confidence in giving constructive feedback
• Enhancing educational interactions with realistic AI mentee behaviors

Innovation

• Role-playing with AI agents for feedback practice
• LLM-based agents simulating mentee idea development
• System enabling mentor role-play and feedback reflection