On Semiotic-Grounded Interpretive Evaluation of Generative Art

2026-04-09

AI Summary

Existing methods for evaluating generative art predominantly focus on image quality or literal alignment with textual prompts, often overlooking the deeper semiotic meanings embedded in the artwork. This work addresses this gap by introducing SemJudge, a novel evaluation framework grounded in Peircean semiotics, which incorporates the triadic modalities of iconicity, symbolism, and indexicality for the first time in this domain. SemJudge models the cascading process of meaning construction from text prompt to generated image through a Hierarchical Semiotic Graph (HSG), enabling a nuanced interpretation of the artist’s intent. Evaluated on interpretive fine-art benchmarks, SemJudge substantially outperforms existing evaluators, and user studies confirm its capacity to produce richer, more insightful artistic interpretations.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Interpretation is essential to deciphering the language of art: audiences communicate with artists by recovering meaning from visual artifacts. However, current Generative Art (GenArt) evaluators remain fixated on surface-level image quality or literal prompt adherence, failing to assess the deeper symbolic or abstract meaning intended by the creator. We address this gap by formalizing a Peircean computational semiotic theory that models Human-GenArt Interaction (HGI) as cascaded semiosis. This framework reveals that artistic meaning is conveyed through three modes - iconic, symbolic, and indexical - yet existing evaluators operate heavily within the iconic mode, remaining structurally blind to the latter two. To overcome this structural blindness, we propose SemJudge. This evaluator explicitly assesses symbolic and indexical meaning in HGI via a Hierarchical Semiosis Graph (HSG) that reconstructs the meaning-making process from prompt to generated artifact. Extensive quantitative experiments show that SemJudge aligns more closely with human judgments than prior evaluators on an interpretation-intensive fine-art benchmark. User studies further demonstrate that SemJudge produces deeper, more insightful artistic interpretations, thereby paving the way for GenArt to move beyond the generation of"pretty"images toward a medium capable of expressing complex human experience. Project page: https://github.com/songrise/SemJudge.

Problem

• Generative Art
• interpretation
• semiotics
• symbolic meaning
• artistic evaluation

Innovation

• computational semiotics
• generative art evaluation
• Peircean semiosis
• symbolic meaning
• human-GenArt interaction