Meta-Reinforcement Learning via Evolution for Multi-Objective Combinatorial Supply Chain Optimisation

2026-06-20

AI Summary

This study addresses the challenges of poor policy adaptability and insufficient diversity in the Pareto front arising from dynamic environmental shifts and preference changes in multi-objective combinatorial supply chain optimization. To tackle these issues, the authors propose a population-based meta-reinforcement learning framework that, for the first time, integrates evolutionary algorithms into the scalarization weight space of meta-reinforcement learning. The approach synergistically combines multi-objective decomposition, gradient-based meta-learning, and evolutionary operators—selection, crossover, and mutation—with population evolution jointly guided by hypervolume and entropy contributions. Experimental results demonstrate that the proposed method achieves a 32% improvement in hypervolume on complex supply chain scenarios, significantly enhancing the diversity, uniformity, and cross-task adaptability of the Pareto solution set, while attaining the lowest average Hausdorff distance among compared approaches.

Abstract

Meta-reinforcement learning is a promising approach to multi-objective optimisation because it enables rapid policy adaptation across changing environments and preference settings. However, conventional few-shot methods usually fine-tune from a single shared meta-policy, which can reduce solution diversity and limit exploration of the Pareto front, especially in high-dimensional combinatorial problems such as supply chain optimisation. We propose a population-based Meta-reinforcement learning framework that combines decomposition with evolutionary search in scalarisation weight space. The framework maintains a population of weight vectors, each associated with a distinct meta-policy trained through gradient-based meta-learning, and iteratively refines this population through elitist selection, crossover, and mutation guided by hypervolume and entropy contributions. We evaluate the method in a multi-objective supply chain setting with conflicting economic, environmental, and social goals, and further test its generality on standard reinforcement learning problems. The results show that the proposed approach yields more diverse, better distributed Pareto front approximations, improves cross-task adaptation, increases hypervolume by up to 32\% over Meta-multi-objective reinforcement learning in the complex case, and attains the lowest average Hausdorff distance among all compared methods.

Problem

• Meta-Reinforcement Learning
• Multi-Objective Optimisation
• Combinatorial Optimisation
• Supply Chain Optimisation
• Pareto Front

Innovation

• population-based meta-reinforcement learning
• evolutionary search
• scalarisation weight space
• Pareto front diversity
• multi-objective supply chain optimisation