ToMigo: Interpretable Design Concept Graphs for Aligning Generative AI with Creative Intent

2026-02-05

AI Summary

This work addresses the challenge that generative AI often fails to produce desired creative outputs due to ambiguous user intent and the absence of intuitive, non-expert-friendly guidance mechanisms. To bridge this gap, the authors propose an editable and interpretable design concept graph as an intermediary representation, where nodes denote purposes, content, or styles, and edges encode explainable relationships. Leveraging a multimodal large language model, the system jointly infers and dynamically aligns design goals from both images and text. Users can interactively edit the concept graph and engage in reflective dialogue to intervene in the AI’s reasoning process in real time. User studies demonstrate that this approach significantly improves intent alignment, enhances user agency, and effectively supports the iterative evolution and realization of creative ideas.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: Generation

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Generative AI often produces results misaligned with user intentions, for example, resolving ambiguous prompts in unexpected ways. Despite existing approaches to clarify intent, a major challenge remains: understanding and influencing AI's interpretation of user intent through simple, direct inputs requiring no expertise or rigid procedures. We present ToMigo, representing intent as design concept graphs: nodes represent choices of purpose, content, or style, while edges link them with interpretable explanations. Applied to graphic design, ToMigo infers intent from reference images and text. We derived a schema of node types and edges from pre-study data, informing a multimodal large language model to generate graphs aligning nodes externally with user intent and internally toward a unified design goal. This structure enables users to explore AI reasoning and directly manipulate the design concept. In our user studies, ToMigo received high alignment ratings and captured most user intentions well. Users reported greater control and found interactive features-editable graphs, reflective chats, concept-design realignment-useful for evolving and realizing their design ideas.

Problem

• generative AI
• creative intent
• intent alignment
• design concept
• user-AI interaction

Innovation

• design concept graphs
• intent alignment
• interpretable AI
• multimodal LLM
• creative intent