Harvesting energy from turbulent winds with Reinforcement Learning

2024-12-18

arXiv.org

AI Summary

This study addresses the poor robustness of airborne wind energy (AWE) systems in turbulent atmospheric boundary layers, where conventional controllers suffer from sensitivity to modeling inaccuracies and reliance on predefined trajectories. To overcome these limitations, we propose a model-free deep reinforcement learning (DRL) control framework. Specifically, we employ the proximal policy optimization (PPO) algorithm trained within a high-fidelity turbulent wind field simulator, and design a lightweight perception policy that operates solely on local state observations—namely, vehicle attitude and relative wind velocity—enabling end-to-end adaptive control. Our key contribution is the first application of model-free RL to AWE control, eliminating dependence on prior aerodynamic models or fixed flight paths. Experimental results demonstrate that the proposed controller significantly improves energy output stability and long-term power generation efficiency under highly uncertain wind conditions, outperforming traditional model predictive control (MPC) in both robustness and cross-wind-condition generalization.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Reinforcement Learning
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Airborne Wind Energy (AWE) is an emerging technology designed to harness the power of high-altitude winds, offering a solution to several limitations of conventional wind turbines. AWE is based on flying devices (usually gliders or kites) that, tethered to a ground station and driven by the wind, convert its mechanical energy into electrical energy by means of a generator. Such systems are usually controlled by manoeuvering the kite so as to follow a predefined path prescribed by optimal control techniques, such as model-predictive control. These methods are strongly dependent on the specific model at use and difficult to generalize, especially in unpredictable conditions such as the turbulent atmospheric boundary layer. Our aim is to explore the possibility of replacing these techniques with an approach based on Reinforcement Learning (RL). Unlike traditional methods, RL does not require a predefined model, making it robust to variability and uncertainty. Our experimental results in complex simulated environments demonstrate that AWE agents trained with RL can effectively extract energy from turbulent flows, relying on minimal local information about the kite orientation and speed relative to the wind.

Problem

• Harness energy from turbulent winds
• Replace traditional control with Reinforcement Learning
• Improve Airborne Wind Energy efficiency

Innovation

• Reinforcement Learning replaces traditional control
• RL adapts to unpredictable turbulent conditions
• AWE extracts energy with minimal local information