🤖 AI Summary
Despite superior sampling efficiency, Bayesian optimization and evolutionary algorithms are often eschewed in practice in favor of less efficient hyperparameter optimization (HPO) methods like grid search.
Method: Through 20 in-depth interviews and a survey of 49 ML practitioners—analyzed via thematic coding and qualitative analysis—we systematically construct the first theory-driven framework explaining HPO method selection.
Contribution/Results: We identify seven core motivational factors (e.g., interpretability, collaborative requirements) and five contextual constraints (e.g., computational resource limitations). Crucially, non-technical human factors—including team workflows and debugging ease—are revealed as decisive determinants of tool adoption, addressing a critical gap in human-centered AutoML research. Our findings have directly informed the redesign of multiple AutoML platforms, improving interactive logic and default strategies to support user-centered, context-adaptive HPO tools.
📝 Abstract
Programmatic hyperparamter optimization (HPO) methods, such as Bayesian optimization and evolutionary algorithms, show high sampling efficiency in finding optimal hyperparameter configurations in development of machine learning (ML) models. Yet, practitioners often use less sample-efficient HPO methods, such as grid search, which often results in under-optimized ML models. As a reason for this behavior, we suspect practitioners choose HPO methods based on different motives. Practitioner motives, however, still need to be clarified to enhance user-centered development of HPO tools. To uncover practitioner motives to use different HPO methods, we conducted 20 semi-structured interviews and an online questionnaire with 49 ML experts. By presenting main goals (e.g., increase ML model understanding) and contextual factors affecting practitioners' selection of HPO methods (e.g., available computer resources), this study offers a conceptual foundation to better understand why practitioners use different HPO methods, supporting design of more user-centered and context-adaptive HPO tools in automated ML.
