FLOW-BENCH: Towards Conversational Generation of Enterprise Workflows

2025-05-16

AI Summary

This study addresses the automatic translation of natural language instructions into executable business process models (e.g., BPMN/DMN) to enhance the reliability and reproducibility of Business Process Automation (BPA). To this end, we introduce FLOW-BENCH—the first high-quality, BPA-oriented evaluation benchmark—and propose FLOW-GEN, a two-stage generation method: first, leveraging multi-scale large language models (LLMs) to map natural language into an interpretable, verifiable Python-style intermediate representation; second, applying semantic mapping to produce standardized BPMN/DMN models. FLOW-GEN integrates natural language understanding, Python-syntax-driven modeling, and rigorous BPMN/DMN semantic alignment. Comprehensive experiments across eight LLMs demonstrate that FLOW-GEN significantly improves generation accuracy, while FLOW-BENCH enables robust, comparable model evaluation. Together, they establish a novel paradigm and a reliable foundation for NL-to-workflow synthesis.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Business process automation (BPA) that leverages Large Language Models (LLMs) to convert natural language (NL) instructions into structured business process artifacts is becoming a hot research topic. This paper makes two technical contributions -- (i) FLOW-BENCH, a high quality dataset of paired natural language instructions and structured business process definitions to evaluate NL-based BPA tools, and support bourgeoning research in this area, and (ii) FLOW-GEN, our approach to utilize LLMs to translate natural language into an intermediate representation with Python syntax that facilitates final conversion into widely adopted business process definition languages, such as BPMN and DMN. We bootstrap FLOW-BENCH by demonstrating how it can be used to evaluate the components of FLOW-GEN across eight LLMs of varying sizes. We hope that FLOW-GEN and FLOW-BENCH catalyze further research in BPA making it more accessible to novice and expert users.

Problem

• Convert natural language to structured business workflows
• Evaluate NL-based business process automation tools
• Facilitate BPA research with datasets and LLM approaches

Innovation

• FLOW-BENCH dataset pairs NL with process definitions
• FLOW-GEN uses LLMs for NL-to-Python conversion
• Converts Python to BPMN and DMN automatically