human-in-the-loop methods

To address the lack of formal modeling support for human–AI collaborative workflows, this paper proposes a BPMN extension tailored for hybrid collaboration between humans and large language model (LLM)-based agents. We introduce the first BPMN metamodel and corresponding graphical notation explicitly supporting human–agent coordination, formally capturing task responsibility assignment, decision logic ownership, and execution strategy binding—thereby bridging a critical expressiveness gap in existing business process modeling languages for mixed-agent scenarios. Furthermore, we design a domain-specific language (DSL) and implement an open-source web-based modeling tool using TypeScript and React, enabling visual specification and formal verification of collaborative processes. Empirical evaluation through case studies demonstrates that the approach achieves strong expressive power, operational feasibility, and practical applicability in real-world human–LLM workflow engineering.

To address the weak dynamic workflow adaptability and poor fault tolerance of multi-agent systems in complex tasks, this paper proposes an evolvable workflow modeling and runtime re-planning framework based on Activity-on-Vertex (AOV) graphs. We pioneer the representation of workflows as dynamic AOV graphs, integrating dependency analysis with parallelism quantification to assess task dependency complexity and enable modular decoupling. Coupled with large language model (LLM)-based agents, the framework supports real-time rescheduling and autonomous error recovery guided by historical execution performance. Experimental results demonstrate significant improvements: +32.7% in task execution efficiency, +28.4% in goal achievement rate, and enhanced fault tolerance—enabling adaptive adjustment of highly concurrent subtasks under dynamic conditions.

Multi-agent workflow design suffers from an expansive design space, heavy reliance on expert intuition, and a tendency to converge prematurely to suboptimal solutions or engage in inefficient trial-and-error. To address these challenges, this paper proposes a three-layer progressive design-space modeling framework that decouples task planning, agent orchestration, and optimization—integrated with a design pattern library and context-aware recommendation algorithms—to realize an interactive, visual tool supporting in-situ guidance and multi-criteria comparison. Our contributions are threefold: (1) the first systematic, structured formalization of the multi-agent design space; (2) visualization-enabled cognitive scaffolding that closes the “perceive–reason–decide” loop; and (3) context-driven real-time recommendations that enhance both design quality and efficiency. User studies and case validations demonstrate significant reductions in design cycle time, improved systematicity in solution exploration, and higher recommendation adoption rates.

Enterprise operational workflows are notoriously difficult to automate end-to-end due to their heavy reliance on human intervention and limited adaptability to change. This work proposes the first action-centric workflow graph framework, which achieves automated construction, execution, and evolution through a three-stage pipeline: structured workflow graphs are extracted from human operation traces, executed via multi-agent online traversal, and continuously optimized in a closed loop using an Adaptive Traversal Reinforcement (ATR) mechanism. Integrating large-scale offline graph construction, graph-guided retrieval, and large language model reasoning, the approach was deployed across four cloud database services. It substantially outperforms the Trace-RAG baseline in coverage breadth, factual accuracy, and diagnostic throughput, achieving an expert blind-review score of 4.95 out of 5.

To address the insufficient quality and reliability of LLM-generated workflows for complex, multi-intent user queries, this paper proposes Opus, a prompt-based intent framework that introduces a reproducible and customizable intent-capture layer between natural-language queries and workflow generation. Opus decomposes mixed intents into structured intent objects through integrated signal extraction, structured parsing, and intent-driven generation. Its core innovations include formal definitions of “workflow signals” and “structured intents,” along with a lightweight intent abstraction mechanism. Evaluated on a benchmark of 1,000 synthetically generated multi-intent queries, Opus significantly improves the logical coherence, semantic consistency, and semantic similarity of generated workflows—particularly under high-complexity conditions.

Automatically constructing high-quality, reusable skills from heterogeneous, fragmented interaction traces—often missing critical security behaviors—is highly challenging. This work proposes the W2S framework, which introduces a novel intermediate representation called RWSA to decouple skills into workflow structure, execution semantics, and runtime attachments, thereby enabling task decomposition, control-flow modeling, verification, rollback, and state management. W2S achieves efficient skill construction through trajectory segmentation, local skill draft generation, structural alignment, branch fusion, redundancy compression, and confidence-aware retention. Experimental evaluation across 70 skills demonstrates that W2S improves behavioral replay consistency by 10.5% compared to baseline approaches based on summarization and prompting.

This study addresses the limitations of traditional expertise theories, which predominantly conceptualize expertise as a static accumulation of knowledge or skills and thus fail to account for its dynamic cognitive mechanisms. To overcome this, the work proposes a “workflow cognition” framework that reconceptualizes expertise as an emergent cognitive phenomenon arising from the recursive coupling and continuous co-evolution of thought flows and workflow patterns through sustained practice. Drawing on theoretical modeling and cross-domain case comparisons—spanning craftsmanship, creative production, education, and leadership—the study constructs a dynamic cognitive ontology of expertise, offering the first systematic account of the internal grammar linking tacit cognition, aesthetic judgment, and expert workflows. This framework provides both a theoretical foundation and a computational modeling pathway for representing human expertise in AI-enhanced expert systems.

This study addresses the lack of standardized and automated case planning processes in medical social work, which currently relies heavily on individual practitioner experience and suffers from inefficiency. The authors propose a model-agnostic, open-source large language model (LLM) workflow that systematically integrates established social work practice frameworks into LLM prompt design for the first time. The approach decomposes case planning into six sequential stages—assessment, problem analysis, goal setting, intervention planning, risk anticipation, and outcome evaluation—and combines structured client profiling with staged prompt engineering to generate professional, reviewable draft assessment forms and service plans. Designed to be compatible across multiple LLM platforms, the framework ensures cross-model reproducibility, and its code has been publicly released to provide a standardized tool for advancing intelligent support in medical social work.

Existing Model Cards and Data Cards describe only static models and datasets, lacking documentation of the execution context surrounding generation, transformation, and evaluation processes—thereby limiting reproducibility and bias analysis. This work proposes Workflow Cards, which extend the structured documentation paradigm to dynamic workflow executions for the first time. Built upon provenance data, Workflow Cards generate machine-readable, structured summaries interpretable by both humans and large language models (LLMs), and incorporate a template designed to answer typical execution-related questions. Experimental results demonstrate that Workflow Cards substantially enhance understanding of workflow executions compared to schema-based query interfaces, nearly doubling answer quality and achieving superior performance under both LLM-as-a-Judge and human evaluations.

This work addresses the limitations of existing agent workflows, which predominantly rely on abstract structures from large language models and lack genuine tool integration, resulting in poor usability and stability. To overcome this, we propose FlowScout, a novel framework that explicitly models real-world tool invocations as nodes in a directed graph. FlowScout integrates tool coordination skeleton mining with a Monte Carlo Tree Search mechanism guided by execution feedback to automatically optimize workflow topology. Experimental results across four task domains demonstrate that FlowScout significantly outperforms baseline methods—including PM4Py, ReAct, and AFlow—with at least a 92.69% improvement in tool invocation accuracy, a minimum 17.66% gain in execution quality, and enhanced runtime stability.