Towards Resource-Efficient Compound AI Systems

2025-01-28

AI Summary

Contemporary composite AI systems suffer from low resource utilization, tight coupling between logical specification and execution, and inherent trade-offs between efficiency and quality in multi-component (e.g., models, retrievers, tools) orchestration. This paper proposes a declarative workflow programming model coupled with an adaptive runtime system to decouple application logic from resource management, enabling joint optimization of workflow orchestration and cluster scheduling. Key contributions include: (1) dynamic resource-aware scheduling, (2) runtime-adaptive decision-making mechanisms, and (3) a modular, configurable execution environment. Implemented as the prototype system Murakkab, our approach achieves a 3.4× reduction in workflow completion time and a 4.5× improvement in energy efficiency—while preserving task quality—thereby significantly enhancing resource efficiency and scalability of composite AI systems.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Compound AI Systems, integrating multiple interacting components like models, retrievers, and external tools, have emerged as essential for addressing complex AI tasks. However, current implementations suffer from inefficient resource utilization due to tight coupling between application logic and execution details, a disconnect between orchestration and resource management layers, and the perceived exclusiveness between efficiency and quality. We propose a vision for resource-efficient Compound AI Systems through a emph{declarative workflow programming model} and an emph{adaptive runtime system} for dynamic scheduling and resource-aware decision-making. Decoupling application logic from low-level details exposes levers for the runtime to flexibly configure the execution environment and resources, without compromising on quality. Enabling collaboration between the workflow orchestration and cluster manager enables higher efficiency through better scheduling and resource management. We are building a prototype system, called extbf{ extit{Murakkab}}, to realize this vision. Our preliminary evaluation demonstrates speedups up to $sim 3.4 imes$ in workflow completion times while delivering $sim 4.5 imes$ higher energy efficiency, showing promise in optimizing resources and advancing AI system design.

Problem

• Resource-efficient AI
• Complex problem solving
• AI system optimization

Innovation

• Murakkab
• Efficient Resource Utilization
• Enhanced Processing Speed and Energy Efficiency