Operationalizing machine learning models

Abstract

Deploying machine learning (ML) models to production with the same level of rigor and automation as traditional software systems has shown itself to be a non-trivial task, requiring extra care and infrastructure to deal with the additional challenges. Although many studies focus on adapting ML software engineering (SE) approaches and techniques, few studies have summarized the status and challenges of operationalizing ML models. Model operationalization encompasses all steps after model training and evaluation, including packaging the model in a format appropriate for deployment, publishing to a model registry or storage, integrating the model into a broader software system, serving, and monitoring. This study is the first systematic literature review investigating the techniques, tools, and infrastructures to operationalize ML models. After reviewing 24 primary studies, the results show that there are a number of tools for most use cases to operationalize ML models and cloud deployment in particular. The review also revealed several research opportunities, such as dynamic model-switching, continuous model-monitoring, and efficient edge ML deployments.