The Value of Certifying Software Release Readiness: An Exploratory Study of Certification for a Critical System at JPL

Abstract

A software release is a decision to deliver code to an organization outside of the development team usually for testing or operational purposes. For critical systems this can be a risky decision where failure to pass a test or holding up the project schedule can have a major impact. The release decision is primarily based on the understanding on the level of quality the software currently has (be it high quality, low quality, or unknown). But for large, complex systems, determining the level of quality with high confidence is a challenge. A poor understanding of the confidence in the quality level increases decision risk leading potentially to a bad release decision that possibly could have been avoided had the confidence in the quality been better known. Certification of release readiness attempts to address this risk by building confidence in the quality level. But this comes at a cost and the relationship between certification and decision risk reduction has not been well understood. This work describes our experience investigating the value of certification and our efforts to improve the mandated software readiness certification record (SRCR) process. A well known critical system at JPL is used as a case study to exemplify this effort.