From Elements to Design: A Layered Approach for Automatic Graphic Design Composition

2024-12-27

AI Summary

Existing automated graphic design models suffer from narrow task coverage and insufficient hierarchical modeling capabilities, hindering effective handling of element ordering and global composition. To address this, we propose the first framework that integrates hierarchical design principles into a large multimodal model (LMM), enabling semantic-driven hierarchical planning: it progressively predicts layout attributes layer-by-layer and generates rendered outputs, supporting cross-layer contextual awareness and inter-layer rendering feedback. Our method requires no task-specific fine-tuning, yet unifies diverse subtasks—including element composition, resolution adaptation, and semantic content filling—within a single architecture. On automated graphic composition, it significantly improves generation quality, controllability, and generalization. It enables design variant generation and resolution-adaptive output, consistently outperforming specialized baseline models across multiple quantitative metrics.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

In this work, we investigate automatic design composition from multimodal graphic elements. Although recent studies have developed various generative models for graphic design, they usually face the following limitations: they only focus on certain subtasks and are far from achieving the design composition task; they do not consider the hierarchical information of graphic designs during the generation process. To tackle these issues, we introduce the layered design principle into Large Multimodal Models (LMMs) and propose a novel approach, called LaDeCo, to accomplish this challenging task. Specifically, LaDeCo first performs layer planning for a given element set, dividing the input elements into different semantic layers according to their contents. Based on the planning results, it subsequently predicts element attributes that control the design composition in a layer-wise manner, and includes the rendered image of previously generated layers into the context. With this insightful design, LaDeCo decomposes the difficult task into smaller manageable steps, making the generation process smoother and clearer. The experimental results demonstrate the effectiveness of LaDeCo in design composition. Furthermore, we show that LaDeCo enables some interesting applications in graphic design, such as resolution adjustment, element filling, design variation, etc. In addition, it even outperforms the specialized models in some design subtasks without any task-specific training.

Problem

• Automatic Graphic Design
• Design Composition
• Hierarchy Awareness

Innovation

• LaDeCo
• Hierarchical Design
• Automatic Graphic Design