PRISM: Learning Design Knowledge from Data for Stylistic Design Improvement

2026-01-16

AI Summary

This work addresses the challenge non-expert users face in efficiently producing style-consistent modifications in graphic design, a task hindered by existing vision-language models that generalize design styles too broadly and diverge from real-world design practice. To bridge this gap, the authors propose a style-aware design refinement approach that constructs a structured knowledge base from authentic design data, captures style diversity through clustering high-variance design samples, and distills actionable design rules. During inference, relevant knowledge is retrieved to guide generation. This method is the first to explicitly model and integrate designers’ implicit stylistic knowledge, enabling stylized edits aligned with professional design practices. Evaluated on the Crello dataset, it achieves a style-alignment mean rank of 1.49 (closer to 1 is better), significantly outperforming baseline methods, and user studies confirm that professional designers strongly prefer its outputs.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Graphic design often involves exploring different stylistic directions, which can be time-consuming for non-experts. We address this problem of stylistically improving designs based on natural language instructions. While VLMs have shown initial success in graphic design, their pretrained knowledge on styles is often too general and misaligned with specific domain data. For example, VLMs may associate minimalism with abstract designs, whereas designers emphasize shape and color choices. Our key insight is to leverage design data -- a collection of real-world designs that implicitly capture designer's principles -- to learn design knowledge and guide stylistic improvement. We propose PRISM (PRior-Informed Stylistic Modification) that constructs and applies a design knowledge base through three stages: (1) clustering high-variance designs to capture diversity within a style, (2) summarizing each cluster into actionable design knowledge, and (3) retrieving relevant knowledge during inference to enable style-aware improvement. Experiments on the Crello dataset show that PRISM achieves the highest average rank of 1.49 (closer to 1 is better) over baselines in style alignment. User studies further validate these results, showing that PRISM is consistently preferred by designers.

Problem

• stylistic design improvement
• natural language instructions
• design knowledge
• visual language models
• graphic design

Innovation

• design knowledge learning
• stylistic design improvement
• visual language models
• knowledge-guided generation
• cluster-based summarization