EIA-SEC: Improved Actor-Critic Framework for Multi-UAV Collaborative Control in Smart Agriculture

2025-12-21

AI Summary

This paper addresses the trajectory planning problem for multi-UAV cooperative data collection, image acquisition, and communication in smart agriculture. It formulates the problem as a Markov decision process and proposes the Elite Imitation Actor–Shared Ensemble Critic (EIA-SEC) framework. Methodologically, EIA-SEC innovatively integrates elite demonstration learning with a two-level critic architecture: an upper-level shared ensemble critic mitigates value overestimation and enhances generalization, while a lower-level local critic enables distributed cooperative value estimation; additionally, behavior cloning is incorporated to reduce exploration overhead. Experimental results demonstrate that EIA-SEC significantly outperforms state-of-the-art reinforcement learning methods in reward performance, training stability, and convergence speed. The framework thus establishes an efficient and robust decision-making paradigm for agricultural multi-UAV coordination.

Technology Category

• Multiagent Systems: Multiagent Learning
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The widespread application of wireless communication technology has promoted the development of smart agriculture, where unmanned aerial vehicles (UAVs) play a multifunctional role. We target a multi-UAV smart agriculture system where UAVs cooperatively perform data collection, image acquisition, and communication tasks. In this context, we model a Markov decision process to solve the multi-UAV trajectory planning problem. Moreover, we propose a novel Elite Imitation Actor-Shared Ensemble Critic (EIA-SEC) framework, where agents adaptively learn from the elite agent to reduce trial-and-error costs, and a shared ensemble critic collaborates with each agent's local critic to ensure unbiased objective value estimates and prevent overestimation. Experimental results demonstrate that EIA-SEC outperforms state-of-the-art baselines in terms of reward performance, training stability, and convergence speed.

Problem

• Develops a multi-UAV trajectory planning framework for smart agriculture tasks
• Proposes an actor-critic method to reduce learning costs and improve stability
• Enhances collaborative control for data collection and communication using UAVs

Innovation

• Elite imitation learning reduces trial-and-error costs
• Shared ensemble critic prevents overestimation bias
• Framework improves reward, stability, and convergence speed