1997 · 29 citations · 29 references
EngineeringQuality ManagementSystems EngineeringPerformance AssessmentQuality EngineeringQuality ImprovementPerformance EvaluationProcess Capability AnalysisReliabilitySoftware MetricsSoftware Quality AssuranceTest ReliabilityAssessor ReliabilityInter-rater AgreementInter-rater ReliabilityInterrater AgreementInterrater ReliabilityProcess CapabilityRater AgreementSoftware Process Improvement
The reliability of process assessments has received some study in the recent past, much of it being conducted within the context of the SPICE (Software Process Improvement and Capability dEtermination) trials. In this paper, we build upon this work by evaluating the reliability of ratings on each of the practices that make up the SPICE capability dimension. The type of reliability that we evaluate is inter-rater agreement: the agreement amongst independent assessors' capability ratings. Inter-rater agreement was found to be generally high. We also identify one particular practice that exhibits low agreement in its ratings.
29
The Measurement of Observer Agreement for Categorical Data
J. Richard Landis, Gary G. Koch · Biometrics · 1977 · 78.9K citations · Full text