Empowering Scientific Workflows with Federated Agents

2025-05-08

AI Summary

Existing agent systems struggle to achieve deep integration with federated scientific research infrastructures—such as high-performance computing (HPC) systems, experimental apparatus, and data warehouses—thereby limiting their practical applicability in scientific computing. To address this, we propose Academy, a novel middleware that introduces the first federated agent architecture specifically designed for scientific computation. Academy enables heterogeneous resource orchestration, stateful execution, tight coupling with physical experimental equipment, and dynamic adaptation. Its design incorporates modular components, an asynchronous execution engine, stateful agent abstractions, cross-domain coordination protocols, and high-throughput dataflow management—collectively supporting end-to-end autonomous collaboration in scientific workflows. Microbenchmarking demonstrates Academy’s high performance and strong scalability within HPC environments. Furthermore, empirical validation across materials discovery, decentralized learning, and information extraction confirms its capability to coordinate agents across multiple distributed HPC systems.

Technology Category

• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Agentic systems, in which diverse agents cooperate to tackle challenging problems, are exploding in popularity in the AI community. However, the agentic frameworks used to build these systems have not previously enabled use with research cyberinfrastructure. Here we introduce Academy, a modular and extensible middleware designed to deploy autonomous agents across the federated research ecosystem, including HPC systems, experimental facilities, and data repositories. To meet the demands of scientific computing, Academy supports asynchronous execution, heterogeneous resources, high-throughput data flows, and dynamic resource availability. It provides abstractions for expressing stateful agents, managing inter-agent coordination, and integrating computation with experimental control. We present microbenchmark results that demonstrate high performance and scalability in HPC environments. To demonstrate the breadth of applications that can be supported by agentic workflow designs, we also present case studies in materials discovery, decentralized learning, and information extraction in which agents are deployed across diverse HPC systems.

Problem

• Enabling agentic frameworks for research cyberinfrastructure integration
• Supporting scientific computing demands with federated autonomous agents
• Facilitating agent coordination across heterogeneous HPC systems

Innovation

• Modular middleware for federated agent deployment
• Supports asynchronous execution and heterogeneous resources
• Enables agent coordination and experimental control integration