From Fragment to One Piece: A Survey on AI-Driven Graphic Design

2025-03-24

AI Summary

This paper addresses core challenges in AI-Guided Graphic Design (AIGD)—namely, difficulty in interpreting human intent, lack of controllability in generation, and opacity of design decisions—by establishing the first unified taxonomy for AIGD and proposing a “fragment-to-whole” design intent modeling paradigm. Methodologically, it integrates multimodal deep learning, vision-language models (VLMs/LLMs), design graph-structured modeling, and controllable diffusion-based generation, comprehensively covering 12 subtasks including visual understanding, aesthetic semantic modeling, layout analysis, and layout generation. Key contributions include: (1) uncovering mechanistic pathways through which LLMs and multimodal collaboration enhance generation controllability and decision interpretability; (2) identifying six fundamental open challenges and four technical evolution directions; and (3) establishing the work as an authoritative survey and pedagogical benchmark in the AIGD field.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

This survey provides a comprehensive overview of the advancements in Artificial Intelligence in Graphic Design (AIGD), focusing on integrating AI techniques to support design interpretation and enhance the creative process. We categorize the field into two primary directions: perception tasks, which involve understanding and analyzing design elements, and generation tasks, which focus on creating new design elements and layouts. The survey covers various subtasks, including visual element perception and generation, aesthetic and semantic understanding, layout analysis, and generation. We highlight the role of large language models and multimodal approaches in bridging the gap between localized visual features and global design intent. Despite significant progress, challenges remain to understanding human intent, ensuring interpretability, and maintaining control over multilayered compositions. This survey serves as a guide for researchers, providing information on the current state of AIGD and potential future directionsfootnote{https://github.com/zhangtianer521/excellent_Intelligent_graphic_design}.

Problem

• AI techniques for design interpretation and creativity enhancement
• Perception and generation tasks in graphic design
• Challenges in human intent understanding and design control

Innovation

• AI techniques for design interpretation
• Large language models in design
• Multimodal approaches for visual features