Current State of Research on Continuous Experimentation: A Systematic Mapping Study

Abstract

The systematic evaluation of ideas by experiments are the foundation of continuous experimentation. It allows to assess the value of an idea, remove guessing and subjective opinions from the discussion. The enormous interest of it by practitioners and researchers let the body of knowledge consistently grow. New framework, methods and techniques are developed and its application is constantly expanded to new fields like cyber-physical systems or social networks. In this paper we present a systematic mapping study to characterize the current state of research on continuous experimentation. Our study analyzes the following aspects: intensity of research activity and industry-academia collaboration, influential authors and publications, frequent research types and topics, kind of contributions and terms used for continuous experimentation. Our findings show amongst others that the intensity of research activities increases consistently, the collaboration between industry and academia is high and the most cited publications are experience reports from practitioners.