Exploring the Potential of Metacognitive Support Agents for Human-AI Co-Creation

2025-06-15

AI Summary

Generative AI design tools introduce core challenges including ambiguous designer intent articulation, excessive cognitive offloading, and insufficient problem exploration. To address these, this study proposes and empirically validates a novel “metacognitive support agent” paradigm that enhances intention formation and process monitoring in human-AI co-design through reflective guidance. Using a Wizard-of-Oz heuristic prototyping approach, we conducted an exploratory experiment with 20 mechanical designers, integrating behavioral observation and solution evaluation. Results show that the agent-supported group produced significantly more feasible design solutions than the control group (p < 0.01), demonstrating that metacognitive intervention effectively improves human-AI collaborative design quality. Furthermore, this work is the first to reveal differential impacts of distinct guidance strategies on design outcomes—providing both theoretical grounding and actionable design principles for human-centered optimization in intelligent design systems.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Economic aspects and information design of Web, GenAI, and cloud computing

Abstract

Despite the potential of generative AI (GenAI) design tools to enhance design processes, professionals often struggle to integrate AI into their workflows. Fundamental cognitive challenges include the need to specify all design criteria as distinct parameters upfront (intent formulation) and designers' reduced cognitive involvement in the design process due to cognitive offloading, which can lead to insufficient problem exploration, underspecification, and limited ability to evaluate outcomes. Motivated by these challenges, we envision novel metacognitive support agents that assist designers in working more reflectively with GenAI. To explore this vision, we conducted exploratory prototyping through a Wizard of Oz elicitation study with 20 mechanical designers probing multiple metacognitive support strategies. We found that agent-supported users created more feasible designs than non-supported users, with differing impacts between support strategies. Based on these findings, we discuss opportunities and tradeoffs of metacognitive support agents and considerations for future AI-based design tools.

Problem

• Addressing cognitive challenges in AI-integrated design workflows
• Exploring metacognitive agents to enhance reflective GenAI collaboration
• Evaluating support strategies for improved design feasibility

Innovation

• Metacognitive support agents enhance design reflection
• Wizard of Oz study tests multiple support strategies
• Agent-supported users produce more feasible designs