State space exploration in Markov models

Abstract

Performance and dependability analysis is usually based on Markov models. One of the main problems faced by the analyst is the large state space cardinality of the Markov chain associated with the model, which precludes not only the model solution, but also the generation of the transition rate matrix. However, in many real system models, most of the probability mass is concentrated in a small number of states in comparison with the whole state space. Therefore, performability measures may be accurately evaluated from these “high probable” states. In this paper, we present an algorithm to generate the most probable states that is more efficient than previous algorithms in the literature. We also address the problem of calculating measures of interest and show how bounds on some measures can be efficiently calculated.