Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment

2026-08-25

AI Summary

本文针对联邦学习中难以复现、比较和扩展的问题，提出Flower Hub平台，通过将基准测试打包为可执行的应用程序，支持在模拟和实际部署中统一评估。

Abstract

Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Existing evaluations are often tied to custom infrastructure, released as incomplete research code, and conducted primarily in simulation, which limits portability and practical relevance. We present Flower Hub, a platform for publishing, discovering, and executing decentralized and federated applications. We show how it enables reproducible benchmarking by packaging benchmarks as executable, versioned applications with standardized metadata, pinned dependencies, and explicit evaluation workflows. We instantiate this approach with a multi-domain benchmark suite spanning cross-silo and cross-device settings, and including tasks in medical imaging, financial tabular learning, legal instruction tuning, phishing URL detection, and audio tagging. We further demonstrate that the same benchmarking application can run across both simulation and deployment runtimes without changing the application code, enabling unified evaluation across varying learning environments. Beyond model quality, our benchmark design supports system-aware reporting, including runtime and communication metrics. This work advances benchmarking in FL settings from ad hoc code artifacts towards portable, executable, and reusable benchmark applications.

Problem

• Federated Learning
• Benchmarking
• Reproducibility
• Decentralized Data
• Simulation

Innovation

• Federated Learning
• Reproducible Benchmarking
• Decentralized Applications
• Simulation and Deployment
• System-aware Reporting