Problem
Research questions and friction points this paper is trying to address.
Carbon Emissions
Edge-Cloud LLM Systems
Sustainability
Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems
🤖 AI Summary
为减少大语言模型功能调用中的能耗和碳排放，提出一种基于边缘-云架构的碳感知路由框架，利用k-NN预测器估计各层性能并结合实时电网碳强度进行优化调度。
📝 Abstract
Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems. Nevertheless, current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions. We address this sustainability challenge with a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture, combining edge and cloud LLMs on heterogeneous hardware. At its core, a lightweight k-NN predictor operating in a unified semantic-lexical embedding space estimates query-specific accuracy, delay, and power consumption on each edge tier. These predictions are then combined with real-time grid carbon intensity to route every query to the lowest-emission tier capable of executing it successfully. Evaluated on state-of-the-art function-calling benchmarks and LLM families, our framework matches cloud-level accuracy while reducing operational carbon emissions by $4\times$ on average.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Carbon-Aware Routing
Edge-Cloud LLM Systems
Unified Semantic-Lexical Embedding Space
Real-Time Grid Carbon Intensity
🔎 Similar Papers
No similar papers found.
