Evolutionary algorithms meet self-supervised learning: a comprehensive survey

2025-04-09

AI Summary

A systematic survey of the integration between evolutionary machine learning (EML) and self-supervised learning (SSL) is currently lacking, hindering the automated design of label-efficient and robust deep neural networks. To address this gap, we introduce “Evolutionary Self-Supervised Learning” (ESSL) as a novel subfield and propose the first unified taxonomy covering algorithmic paradigms, representation learning, and optimization objectives. Our framework bridges EML techniques—including genetic programming and neural architecture search—with SSL strategies such as contrastive learning and masked modeling. This cross-paradigm synthesis fills a critical survey void, identifies core challenges—including joint optimization difficulty and absence of standardized evaluation metrics—and outlines concrete future research directions. The work establishes both theoretical foundations and practical pathways for reducing annotation dependency, enhancing model generalization, and advancing automated deep learning design. (149 words)

Technology Category

• Machine Learning: Evolutionary Learning
• Search and Optimization: Evolutionary Computation
• Multiagent Systems: Multiagent Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The number of studies that combine Evolutionary Machine Learning and self-supervised learning has been growing steadily in recent years. Evolutionary Machine Learning has been shown to help automate the design of machine learning algorithms and to lead to more reliable solutions. Self-supervised learning, on the other hand, has produced good results in learning useful features when labelled data is limited. This suggests that the combination of these two areas can help both in shaping evolutionary processes and in automating the design of deep neural networks, while also reducing the need for labelled data. Still, there are no detailed reviews that explain how Evolutionary Machine Learning and self-supervised learning can be used together. To help with this, we provide an overview of studies that bring these areas together. Based on this growing interest and the range of existing works, we suggest a new sub-area of research, which we call Evolutionary Self-Supervised Learning and introduce a taxonomy for it. Finally, we point out some of the main challenges and suggest directions for future research to help Evolutionary Self-Supervised Learning grow and mature as a field.

Problem

• Combining evolutionary algorithms with self-supervised learning to automate ML design
• Reducing reliance on labeled data through evolutionary self-supervised learning techniques
• Establishing a taxonomy and addressing challenges in evolutionary self-supervised learning

Innovation

• Combines evolutionary algorithms with self-supervised learning
• Automates deep neural network design processes
• Reduces dependency on labeled training data