Frontiers in Marine Science · 2016 · 61 citations · 52 references
Natural SciencesMarine SciencesOcean AcousticsOcean InstrumentationRemote SensingAnnotationImage AnnotationOceanographic SensorsSeafloor MappingAnnotation Quality AssessmentDigital Pathology AnalysisHabitat MappingImage Sensor TechnologyMarine Object RecognitionSensor TechnologyUnderwater Acoustic SensorsUnderwater Imaging
Marine imaging is transforming into a sensor technology applied for high throughput sampling. In the context of habitat mapping, imaging establishes thereby an important bridge technology regarding the spatial resolution and information content between physical sampling gear (e.g. box corer, multi corer) on the one end and hydro-acoustic sensors on the other end of the spectrum of sampling methods. In contrast to other scientific imaging domains, such as digital pathology, there are no protocols and reports available that guide users (often referred to as observers) in the non-trivial process of assigning semantic categories to whole images, regions or objects of interest, which is referred to as annotation. These protocols are crucial to facilitate image analysis as a robust scientific method. In this article we will review the past observations in manual Marine Image Annotations (MIA) and provide a) a guideline for collecting manual annotations, b) definitions for annotation quality and c) a statistical framework to analyze the performance of human expert annotations and to compare those to computational approaches.
52
The Measurement of Observer Agreement for Categorical Data
J. Richard Landis, Gary G. Koch · Biometrics · 1977 · 78.9K citations · Full text
Understanding interobserver agreement: the kappa statistic.
Anthony J. Viera, Joanne M. Garrett · PubMed · 2005 · 7.1K citations