Sampling Method and Sample Placement

Abstract

for accuracy against the reference data. A widely accepted proThe accuracy of remotely sensed forest stand maps is tra- cedure for comparing these data is the generation of an error ditionally assessed by comparing a sample of the map data matrix (Card, 1982; Congalton et al., 1983; Story and Congalton, with actual ground conditions. Samples most often comprise 1986; Congalton, 1991). clusters of pixels within homogeneous areas, thereby avoiding An error matrix is an especially effective accuracy assessproblems associated with accurately mapping “edges” (e.g., ment tool because it provides a starting point for a series of statransition areas between two forest types). Consequently, they tistical techniques to further examine accuracy (Congalton and may well overestimate accuracy, but the degree of overestima- Green, 1999). One such analytical technique is the Kappa analtion is unknown. This paper examines two important factors ysis, a discrete multivariate technique for comparing error maregarding accuracy assessment that are not well studied: the trices (Congalton et al., 1983; Hudson and Ramm, 1987; Coneffect on estimates of accuracy of (1) the sampling method galton, 1991; Ma and Redmond, 1995; Stehman, 1996; Stehand (2) the exact placement of the samples. Overall accuracy, man, 1999; Congalton and Green, 1999). Kappa analysis, which normalized accuracy, and the KHAT statistic are computed from assumes a multinomial distribution, generates a KHAT statistic error matrices generated from simple random sampling, stra- that measures the difference between actual and chance (or rantified random sampling, and systematic sampling using totally dom) agreement between the map and reference data. It can also random sample placement and samples chosen from homog- be used to test for significant differences between two error eneous areas only. The results indicate that Kappa appears matrices. to be as appropriate to use with systematic sampling and The only sampling method that satisfies Kappa’s assumpstratified random sampling as it is with simple random sam- tion of a multinomial model is simple random sampling. The pling, but suggests that sample placement may have more of effect of other sampling schemes on the outcome of the Kappa an effect on estimates of accuracy than sampling method analysis has not been well studied. In addition, samples are ofalone. ten chosen only if they occur within the interior of homogeneous pixel groupings in order to avoid problems with sam