Performance-influence models for highly configurable systems

TLDR

Complex software systems are highly configurable, yet this configurability complicates performance prediction, optimization, and debugging because the effects of individual options and their interactions are often unknown and combinatorially large. The study proposes a method to derive a performance‑influence model that captures all relevant option effects and interactions for a configurable system. The method integrates machine‑learning techniques with sampling heuristics to construct the model. The approach outperforms standard techniques by explicitly representing option interactions, handling binary and numeric options, incorporating domain knowledge, respecting constraints, and reducing the solution space, and experiments confirm its feasibility and predictive accuracy.

Abstract

Almost every complex software system today is configurable. While configurability has many benefits, it challenges performance prediction, optimization, and debugging. Often, the influences of individual configuration options on performance are unknown. Worse, configuration options may interact, giving rise to a configuration space of possibly exponential size. Addressing this challenge, we propose an approach that derives a performance-influence model for a given configurable system, describing all relevant influences of configuration options and their interactions. Our approach combines machine-learning and sampling heuristics in a novel way. It improves over standard techniques in that it (1) represents influences of options and their interactions explicitly (which eases debugging), (2) smoothly integrates binary and numeric configuration options for the first time, (3) incorporates domain knowledge, if available (which eases learning and increases accuracy), (4) considers complex constraints among options, and (5) systematically reduces the solution space to a tractable size. A series of experiments demonstrates the feasibility of our approach in terms of the accuracy of the models learned as well as the accuracy of the performance predictions one can make with them.