Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms

TLDR

Sequential model‑based (Bayesian) optimization is highly efficient per function evaluation, making it suitable for tuning hyperparameters of slow‑to‑train machine learning models. The paper introduces Hyperopt, demonstrating its use for defining search spaces, serial and parallel minimization, result analysis, and outlines future directions. Hyperopt supplies Python algorithms and parallel infrastructure for hyperparameter optimization, supporting serial and parallel search space exploration and result analysis.

Abstract

Sequential model-based optimization (also known as Bayesian optimization) is one of the most efficient methods (per function evaluation) of function minimization. This efficiency makes it appropriate for optimizing the hyperparameters of machine learning algorithms that are slow to train. The Hyperopt library provides algorithms and parallelization infrastructure for performing hyperparameter optimization (model selection) in Python. This paper presents an introductory tutorial on the usage of the Hyperopt library, including the description of search spaces, minimization (in serial and parallel), and the analysis of the results collected in the course of minimization. The paper closes with some discussion of ongoing and future work.