Discrete Time Reward Models for Homogeneous Semi-Markov Systems

Abstract

Abstract Previously, continuous time Markov and semi-Markow models have been developed for a multi-grade system with a number of transient states and a single absorbing state. Such systems have been studied with Poisson arrivals with known growth. Such models have been applied to manpower planning and the movement of patients through a hospital system. In this paper, we develop a reward model for a discrete time homogeneous semi-Markov system with Poisson arrivals and the same model where the system has growth with known size at each time point. Discrete time is employed here to accommodate a discrete reward structure. Results are obtained for the distribution, and mean of daily rewards of such system at any time, and in steady state; the expected reward in a period of time is also determined. These results are illustrated using data on employees moving through a company hierarchy and data for hospital patients. Key Words: HomogeneityPoisson recruitmentSemi-Markov processStochastic population modelRewards Acknowledgments