Peer-to-Peer Energy Trading in Dairy Farms using Multi-Agent Reinforcement Learning

2025-11-28

AI Summary

To address low renewable energy decentralization efficiency and inadequate peak-valley load regulation in rural dairy farming communities under dynamic conditions, this paper proposes a decentralized optimization framework integrating multi-agent reinforcement learning (MARL) with peer-to-peer (P2P) energy trading. The approach innovatively synergizes Deep Q-Network (DQN) and Proximal Policy Optimization (PPO), augmented by an auction-based market clearing mechanism, price advisory agents, and coordinated load–storage control to enable adaptive distributed decision-making. Experimental evaluation across Irish and Finnish rural dairy farm scenarios demonstrates that DQN reduces electricity procurement costs by 14.2% and 5.16%, respectively, while increasing revenue from surplus energy sales by 7.24% and 12.73%. PPO achieves up to 55.5% peak load reduction; DQN achieves 50.0% (Ireland) and 27.02% (Finland) reductions. The framework significantly enhances energy economic efficiency and system resilience.

Technology Category

• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Distributed Machine Learning & Federated Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

The integration of renewable energy resources in rural areas, such as dairy farming communities, enables decentralized energy management through Peer-to-Peer (P2P) energy trading. This research highlights the role of P2P trading in efficient energy distribution and its synergy with advanced optimization techniques. While traditional rule-based methods perform well under stable conditions, they struggle in dynamic environments. To address this, Multi-Agent Reinforcement Learning (MARL), specifically Proximal Policy Optimization (PPO) and Deep Q-Networks (DQN), is combined with community/distributed P2P trading mechanisms. By incorporating auction-based market clearing, a price advisor agent, and load and battery management, the approach achieves significant improvements. Results show that, compared to baseline models, DQN reduces electricity costs by 14.2% in Ireland and 5.16% in Finland, while increasing electricity revenue by 7.24% and 12.73%, respectively. PPO achieves the lowest peak hour demand, reducing it by 55.5% in Ireland, while DQN reduces peak hour demand by 50.0% in Ireland and 27.02% in Finland. These improvements are attributed to both MARL algorithms and P2P energy trading, which together results in electricity cost and peak hour demand reduction, and increase electricity selling revenue. This study highlights the complementary strengths of DQN, PPO, and P2P trading in achieving efficient, adaptable, and sustainable energy management in rural communities.

Problem

• Optimizing energy distribution in dairy farms using P2P trading
• Reducing electricity costs and peak demand with MARL algorithms
• Improving energy revenue through auction-based market mechanisms

Innovation

• Multi-Agent Reinforcement Learning optimizes P2P energy trading
• Auction-based market clearing with price advisor agent
• DQN and PPO algorithms reduce costs and peak demand