GLOW: Graph-Language Co-Reasoning for Agentic Workflow Performance Prediction

2025-12-11

AI Summary

Existing methods for predicting agent workflow (AW) performance suffer from high execution evaluation costs and struggle to jointly capture topological dependencies and deep semantic logic. To address this, we propose a unified prediction framework that synergistically integrates graph-structured reasoning with linguistic semantics. Our approach innovatively combines a graph-guided instruction-tuned large language model with a graph neural network (GNN), and introduces topology-aware semantic feature extraction alongside latent-space contrastive alignment. By fusing multimodal features and jointly modeling semantic and structural information, our method significantly improves both prediction accuracy and ranking consistency. Evaluated on the FLORA-Bench benchmark, it outperforms all state-of-the-art methods, achieving superior performance in both accuracy and NDCG. This enables low-overhead, highly scalable automated AW generation.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Agentic Workflows (AWs) have emerged as a promising paradigm for solving complex tasks. However, the scalability of automating their generation is severely constrained by the high cost and latency of execution-based evaluation. Existing AW performance prediction methods act as surrogates but fail to simultaneously capture the intricate topological dependencies and the deep semantic logic embedded in AWs. To address this limitation, we propose GLOW, a unified framework for AW performance prediction that combines the graph-structure modeling capabilities of GNNs with the reasoning power of LLMs. Specifically, we introduce a graph-oriented LLM, instruction-tuned on graph tasks, to extract topologically aware semantic features, which are fused with GNN-encoded structural representations. A contrastive alignment strategy further refines the latent space to distinguish high-quality AWs. Extensive experiments on FLORA-Bench show that GLOW outperforms state-of-the-art baselines in prediction accuracy and ranking utility.

Problem

• Predicts performance of Agentic Workflows to reduce evaluation cost
• Captures both topological dependencies and semantic logic in workflows
• Combines graph neural networks with large language models for prediction

Innovation

• Graph-oriented LLM extracts topological semantic features
• Fuses GNN structural representations with LLM reasoning
• Contrastive alignment refines latent space for quality distinction