Reinforcement Learning-assisted Constraint Relaxation for Constrained Expensive Optimization

2026-01-31

AI Summary

This work addresses the challenge of balancing objective optimization and feasible region exploration in expensive black-box constrained optimization, where existing constraint-handling methods rely heavily on manual design. The study proposes the first reinforcement learning–based approach to adaptive constraint relaxation: by formulating a tailored Markov decision process, a deep Q-network dynamically adjusts constraint tolerance based on real-time optimization dynamics, thereby automatically balancing exploitation and exploration without requiring expert knowledge. The method is general-purpose and fully automated. Evaluated on the CEC 2017 constrained optimization benchmark suite under limited function evaluation budgets, it achieves or surpasses the performance of recent top-performing algorithms from CEC/GECCO competitions, demonstrating both effectiveness and strong generalization capability.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Constraint handling plays a key role in solving realistic complex optimization problems. Though intensively discussed in the last few decades, existing constraint handling techniques predominantly rely on human experts'designs, which more or less fall short in utility towards general cases. Motivated by recent progress in Meta-Black-Box Optimization where automated algorithm design can be learned to boost optimization performance, in this paper, we propose learning effective, adaptive and generalizable constraint handling policy through reinforcement learning. Specifically, a tailored Markov Decision Process is first formulated, where given optimization dynamics features, a deep Q-network-based policy controls the constraint relaxation level along the underlying optimization process. Such adaptive constraint handling provides flexible tradeoff between objective-oriented exploitation and feasible-region-oriented exploration, and hence leads to promising optimization performance. We train our approach on CEC 2017 Constrained Optimization benchmark with limited evaluation budget condition (expensive cases) and compare the trained constraint handling policy to strong baselines such as recent winners in CEC/GECCO competitions. Extensive experimental results show that our approach performs competitively or even surpasses the compared baselines under either Leave-one-out cross-validation or ordinary train-test split validation. Further analysis and ablation studies reveal key insights in our designs.

Problem

• Constrained Optimization
• Expensive Optimization
• Constraint Handling
• Reinforcement Learning
• Black-Box Optimization

Innovation

• Reinforcement Learning
• Constraint Handling
• Expensive Optimization
• Adaptive Constraint Relaxation
• Meta-Black-Box Optimization