An experimental survey and Perspective View on Meta-Learning for Automated Algorithms Selection and Parametrization

2025-04-08

AI Summary

The algorithm selection and parameterization (ASP) domain lacks systematic surveys and empirical evaluations. Method: We propose the first standardized, meta-learning–driven ASP framework, built upon the largest ASP benchmark knowledge base to date—comprising 400 datasets and 4 million pre-trained models—and conduct large-scale comparative experiments across eight mainstream classifiers under diverse scenarios. Our evaluation integrates empirical performance modeling (EPM), feature engineering, and statistical significance testing to quantify accuracy, generalizability, and computational efficiency. Contribution/Results: This work delivers the first critical survey balancing methodological rigor with empirical breadth; reveals performance boundaries and applicability conditions of state-of-the-art ASP methods; and establishes a reproducible benchmark and practical selection guide for AutoML research and deployment.

Technology Category

• Machine Learning: Auto ML and Hyperparameter Tuning
• Search and Optimization: Metareasoning and Metaheuristics
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Considerable progress has been made in the recent literature studies to tackle the Algorithms Selection and Parametrization (ASP) problem, which is diversified in multiple meta-learning setups. Yet there is a lack of surveys and comparative evaluations that critically analyze, summarize and assess the performance of existing methods. In this paper, we provide an overview of the state of the art in this continuously evolving field. The survey sheds light on the motivational reasons for pursuing classifiers selection through meta-learning. In this regard, Automated Machine Learning (AutoML) is usually treated as an ASP problem under the umbrella of the democratization of machine learning. Accordingly, AutoML makes machine learning techniques accessible to domain scientists who are interested in applying advanced analytics but lack the required expertise. It can ease the task of manually selecting ML algorithms and tuning related hyperparameters. We comprehensively discuss the different phases of classifiers selection based on a generic framework that is formed as an outcome of reviewing prior works. Subsequently, we propose a benchmark knowledge base of 4 millions previously learned models and present extensive comparative evaluations of the prominent methods for classifiers selection based on 08 classification algorithms and 400 benchmark datasets. The comparative study quantitatively assesses the performance of algorithms selection methods along while emphasizing the strengths and limitations of existing studies.

Problem

• Surveying meta-learning methods for algorithm selection and parametrization
• Lack of comprehensive evaluation of existing ASP approaches
• Proposing benchmark for comparing classifier selection methods

Innovation

• Meta-learning for automated algorithm selection
• AutoML democratizes machine learning accessibility
• Benchmark knowledge base with 4M models