Metaheuristics and Large Language Models Join Forces: Toward an Integrated Optimization Approach

2024-05-28

IEEE Access

AI Summary

Metaheuristics (MHs) for combinatorial optimization on social networks suffer from a lack of problem-specific prior knowledge, limiting their search efficacy. Method: This paper introduces the first large language model (LLM)-driven MH enhancement paradigm: an LLM serves as a structured pattern recognizer, extracting domain knowledge—e.g., community characteristics and influence propagation patterns—from problem descriptions and network topologies via customized prompt engineering; this knowledge dynamically guides neighborhood construction and operator selection within MHs. The approach requires no LLM fine-tuning, integrates seamlessly with mainstream MH frameworks, and yields interpretable, reproducible outputs. Contribution/Results: Evaluated on benchmark social network optimization tasks—including influence maximization and overlapping community detection—the method consistently outperforms existing machine learning–enhanced MHs, achieving an average 12.7% improvement in solution quality. The open-source toolkit OptiPattern is publicly released.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Optimization
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Since the rise of Large Language Models (LLMs) a couple of years ago, researchers in metaheuristics (MHs) have wondered how to use their power in a beneficial way within their algorithms. This paper introduces a novel approach that leverages LLMs as pattern recognition tools to improve MHs. The resulting hybrid method, tested in the context of a social network-based combinatorial optimization problem, outperforms existing state-of-the-art approaches that combine machine learning with MHs regarding the obtained solution quality. By carefully designing prompts, we demonstrate that the output obtained from LLMs can be used as problem knowledge, leading to improved results. Lastly, we acknowledge LLMs’ potential drawbacks and limitations and consider it essential to examine them to advance this type of research further. Our method can be reproduced using a tool available at: https://github.com/camilochs/optipattern.

Problem

• Integrate LLMs with metaheuristics for optimization.
• Use LLMs as pattern recognition tools in algorithms.
• Improve solution quality in combinatorial optimization problems.

Innovation

• LLMs enhance metaheuristics via pattern recognition
• Hybrid method improves combinatorial optimization solutions
• Prompt design optimizes LLM output for problem-solving