Explainability-in-Action: Enabling Expressive Manipulation and Tacit Understanding by Bending Diffusion Models in ComfyUI

2025-08-10

AI Summary

Large generative models—particularly text-to-image diffusion models—are structurally opaque, hindering artists’ intuitive understanding and precise control. Method: This study introduces the “interpretability-in-practice” framework, shifting eXplainable AI (XAI) from a transparency-oriented tool toward a reflective methodology supporting artistic practice. By integrating model decomposition, feature visualization, and interactive parameter tuning plugins into the ComfyUI node-based interface, artists gain sustained, intuitive access to internal model mechanisms and can perform structural interventions throughout extended creative workflows. Results: Empirical evaluation demonstrates significant improvements in artists’ functional intuition about model components and their ability to manipulate them, thereby enhancing expressive freedom and practical autonomy. The core contribution lies in redefining the XAI paradigm: anchoring interpretability in craft practice to transform it into plasticity, modifiability, and continuous learning.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Explainable AI (XAI) in creative contexts can go beyond transparency to support artistic engagement, modifiability, and sustained practice. While curated datasets and training human-scale models can offer artists greater agency and control, large-scale generative models like text-to-image diffusion systems often obscure these possibilities. We suggest that even large models can be treated as creative materials if their internal structure is exposed and manipulable. We propose a craft-based approach to explainability rooted in long-term, hands-on engagement akin to Schön's "reflection-in-action" and demonstrate its application through a model-bending and inspection plugin integrated into the node-based interface of ComfyUI. We demonstrate that by interactively manipulating different parts of a generative model, artists can develop an intuition about how each component influences the output.

Problem

• Enhancing artistic engagement with explainable AI in creative contexts
• Exposing and manipulating internal structures of large generative models
• Developing intuition through interactive model manipulation for artists

Innovation

• Exposing internal structure of large models
• Interactive manipulation in ComfyUI interface
• Craft-based explainability for artistic intuition