A Model for the Measurement of the Runtime Testability of Component-Based Systems

Abstract

Runtime testing is emerging as the solution for the integration and validation of software systems where traditional development-time integration testing cannot be performed, such as Systems of Systems or Service Oriented Architectures.However, performing tests during deployment or in-service time introduces interference problems, such as undesired side-effects in the state of the system or the outside world.This paper presents a qualitative model of runtime testability that complements Binder's classical testability model, and a generic measurement framework for quantitatively assessing the degree of runtime testabilityof a system based on the ratio of what can be tested at runtime vs. what would have been tested during development time.A measurement is devised for the concrete case of architecture-based test coverage, by using a graph model of the system's architecture. Concretely, two testabilitystudies are performed for two component based systems, showing how to measure the runtime testability of a system.