Beyond Semantic Similarity: Open Challenges for Embedding-Based Creative Process Analysis Across AI Design Tools

2026-03-08

AI Summary

Existing embedding-based approaches to analyzing creative processes rely on static semantic similarity, which struggles to capture pivotal transitions in creative trajectories and lacks cross-domain comparability. This work systematically identifies, for the first time, three open challenges inherent in applying embedding methods to creativity analysis. To address these limitations, the paper proposes a novel paradigm that integrates large language models into a context-aware intervention framework for dynamically parsing multimodal design trajectories. By enhancing sensitivity to conversational context, the approach significantly improves the segmentation, representation, and evaluation of creative processes. This advancement lays both a theoretical foundation and a technical pathway toward building analytical frameworks with greater creative sensitivity.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Models for Web evolution
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

AI-based creativity support tools (CSTs) are evaluated through domain-specific metrics, limiting cross-domain comparison of creative processes. Embedding-based protocol analysis offers a potential domain-agnostic analytical layer. However, we argue that fixed embedding similarity can misrepresent creative dynamics: it may not detect creative pivots that occur within superficially similar language, treating shifts in the problem being addressed as continued elaboration. We identify three open challenges stemming from this gap: aligning similarity measures with creative significance, segmenting and representing multimodal design traces, and evaluating agentic systems where embedding-based metrics enter the generation loop and shape agent behavior. We propose context-aware interventions using large language models as a direction for making trace analysis sensitive to session-specific creative dynamics.

Problem

• embedding-based analysis
• creative process
• semantic similarity
• AI creativity support tools
• cross-domain comparison

Innovation

• embedding-based analysis
• creative pivots
• context-aware intervention
• multimodal design traces
• large language models