Tree-structured Parzen estimator: Understanding its algorithm components and their roles for better empirical performance
The Tree-structured Parzen Estimator (TPE), a mainstream Bayesian optimization method, lacks a systematic understanding of the functional roles, impact patterns, and synergistic interactions among its key hyperparameters (e.g., γ, n_startup, n_ei_candidates). Method: We conduct comprehensive ablation studies across diverse benchmark functions to empirically dissect these parameters’ behaviors and dependencies. Based on rigorous empirical analysis, we derive an interpretable and robust default configuration strategy. Contribution/Results: Our recommended configuration significantly outperforms the standard TPE implementation and leading baselines—including SMAC and Hyperopt—across heterogeneous benchmarks, improving both optimization efficiency and stability. All experiments are fully reproducible, with source code publicly released.