ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow Development

2025-06-05

AI Summary

To address the steep learning curve, error-prone model configuration, and workflow design complexity faced by novice users of the ComfyUI platform, this paper proposes a hierarchical multi-agent framework. The framework features a central orchestrator agent that coordinates specialized worker agents, integrated with a custom ComfyUI knowledge graph and domain-specific plugin interfaces (Custom Node API). It enables intelligent node recommendation and one-click generation of interpretable, executable workflows. Experimental evaluation demonstrates an offline accuracy of 92.3% in workflow synthesis and a 68% average reduction in user development time. To our knowledge, this is the first work to deeply integrate a structured-knowledge-driven multi-agent system into a visual AI authoring platform. The approach significantly lowers the entry barrier for beginners while simultaneously enhancing productivity for experienced developers.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Multiagent Planning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

We introduce ComfyUI-Copilot, a large language model-powered plugin designed to enhance the usability and efficiency of ComfyUI, an open-source platform for AI-driven art creation. Despite its flexibility and user-friendly interface, ComfyUI can present challenges to newcomers, including limited documentation, model misconfigurations, and the complexity of workflow design. ComfyUI-Copilot addresses these challenges by offering intelligent node and model recommendations, along with automated one-click workflow construction. At its core, the system employs a hierarchical multi-agent framework comprising a central assistant agent for task delegation and specialized worker agents for different usages, supported by our curated ComfyUI knowledge bases to streamline debugging and deployment. We validate the effectiveness of ComfyUI-Copilot through both offline quantitative evaluations and online user feedback, showing that it accurately recommends nodes and accelerates workflow development. Additionally, use cases illustrate that ComfyUI-Copilot lowers entry barriers for beginners and enhances workflow efficiency for experienced users. The ComfyUI-Copilot installation package and a demo video are available at https://github.com/AIDC-AI/ComfyUI-Copilot.

Problem

• Enhances usability of ComfyUI for AI art creation
• Reduces complexity in workflow design for beginners
• Automates node recommendations and workflow construction

Innovation

• LLM-powered plugin for ComfyUI enhancement
• Hierarchical multi-agent framework for task delegation
• Automated one-click workflow construction