๐ค AI Summary
This work addresses the challenge of configuration tuning, which is often hindered by a lack of interpretability, as existing analytical approaches suffer from limited generalizability and explanatory power. The authors propose DomLand, a novel method that uniquely integrates fitness landscape analysis with domain knowledge to systematically uncover the intrinsic structural properties of configuration spaces. Through a combination of static domain analysis, dynamic performance profiling, and cross-system workload experiments, evaluations across nine systems and 93 workloads reveal that configuration landscapes exhibit strong system-specific characteristics; that core functional options predominantly govern tuning difficultyโfar outweighing resource-related parameters; and that workload effects on tuning outcomes vary significantly across systems. DomLand not only elucidates the root causes behind tuner successes and failures but also offers principled guidance for designing more effective tuning strategies.
๐ Abstract
Configuration tuning for better performance is crucial in quality assurance. Yet, there has long been a mystery on tuners' effectiveness, due to the black-box nature of configurable systems. Prior efforts predominantly adopt static domain analysis (e.g., static taint analysis), which often lacks generalizability, or dynamic data analysis (e.g., benchmarking performance analysis), limiting explainability. In this work, we embrace Fitness Landscape Analysis (FLA) as a bridge between domain knowledge and difficulty of the tuning. We propose Domland, a two-pronged methodology that synergizes the spatial information obtained from FLA and domain-driven analysis to systematically capture the hidden characteristics of configuration tuning cases, explaining how and why a tuner might succeed or fail. This helps to better interpret and contextualize the behavior of tuners and inform tuner design. To evaluate Domland, we conduct a case study of nine software systems and 93 workloads, from which we reveal several key findings: (1) configuration landscapes are inherently system-specific, with no single domain factor (e.g., system area, programming language, or resource intensity) consistently shaping their structure; (2) the core options (e.g., pic-struct of x264), which control the main functional flows, exert a stronger influence on landscape ruggedness (i.e. the difficulty of tuning) compared to resource options (e.g., cpu-independent of x264); (3) Workload effects on landscape structure are not uniformly tied to type or scale. Both contribute to landscape variations, but their impact is system-dependent.