RobustFlow: Towards Robust Agentic Workflow Generation

2025-09-25

AI Summary

Large language models (LLMs) exhibit poor robustness and inconsistent outputs when generating agentic workflows under semantically equivalent but lexically diverse instructions. To address this, we propose a structure-aware preference optimization framework. Our method introduces: (1) a dual-dimensional similarity metric—incorporating both node functionality and topological structure—to quantify workflow structural consistency; and (2) a synonym task description dataset, leveraged alongside structural similarity feedback for preference alignment training. This enables the model to achieve semantic invariance against instruction paraphrasing. In standard evaluations, our approach improves workflow robustness to 70–90%, substantially outperforming baseline methods. The framework provides a verifiable, principled pathway toward enhancing the reliability of LLM-generated agentic workflows, advancing trustworthiness in autonomous agent systems.

Technology Category

• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The automated generation of agentic workflows is a promising frontier for enabling large language models (LLMs) to solve complex tasks. However, our investigation reveals that the robustness of agentic workflow remains a critical, unaddressed challenge. Current methods often generate wildly inconsistent workflows when provided with instructions that are semantically identical but differently phrased. This brittleness severely undermines their reliability and trustworthiness for real-world applications. To quantitatively diagnose this instability, we propose metrics based on nodal and topological similarity to evaluate workflow consistency against common semantic variations such as paraphrasing and noise injection. Subsequently, we further propose a novel training framework, RobustFlow, that leverages preference optimization to teach models invariance to instruction variations. By training on sets of synonymous task descriptions, RobustFlow boosts workflow robustness scores to 70% - 90%, which is a substantial improvement over existing approaches. The code is publicly available at https://github.com/DEFENSE-SEU/RobustFlow.

Problem

• Addresses inconsistent workflow generation from semantically identical instructions
• Proposes metrics to evaluate workflow consistency against semantic variations
• Develops training framework to improve robustness of agentic workflows

Innovation

• Proposes metrics for workflow consistency evaluation
• Introduces RobustFlow training framework for robustness
• Uses preference optimization for instruction variation invariance