Automated Management of Deep Learning Experiments

Abstract

Developing a deep learning model is an iterative, experimental process that produces tens to hundreds of models before arriving at a satisfactory result. While there has been a surge in the number of software tools that aim to facilitate deep learning, the process of managing the models and their artifacts is still surprisingly challenging and time-consuming. Existing model-management solutions are either tailored for commercial platforms or require significant code changes. In this paper, we introduce a lightweight system, named ModelKB, that can automatically extract and manage the model's metadata and provenance information (e.g., the used datasets and hyperparameters). Our overarching goal is to automate the management of deep learning experiments with minimal user intervention. Moreover, ModelKB provides a stepping stone to facilitate model selection and reproducibility.