AI Deployment Architecture: Multi-Case Study for Key Factor Identification

Abstract

Machine learning and deep learning techniques are becoming increasingly popular and critical for companies as part of their systems. However, although the development and prototyping of ML/DL systems are common across companies, the transition from prototype to production-quality deployment models are challenging. One of the key challenges is how to determine the selection of an optimal architecture for AI deployment. Based on our previous research, and to offer support and guidance to practitioners, we developed a framework in which we present five architectural alternatives for AI deployment ranging from centralized to fully decentralized edge architectures. As part of our research, we validated the framework in software-intensive embedded system companies and identified key challenges they face when deploying ML/DL models. In this paper, and to further advance our research on this topic, we identify factors that help practitioners determine what architecture to select for the ML/D L model deployment. For this, we conducted a follow-up study involving interviews and workshops in seven case companies in the embedded systems domain. Based on our findings, we identify three key factors and develop a framework in which we outline how prioritization and trade-offs between these results in certain architecture. The contribution of the paper is threefold. First, we identify key factors critical for AI system deployment. Second, we present the architecture selection framework that explains how prioritization and trade-offs between key factors result in the selection of a certain architecture. Third, we discuss additional factors that mayor may not influence the selection of an optimal architecture.