2023 · 14 citations · 13 references
ComputingComputer ScienceComputer VisionMachine LearningMachine VisionImage SegmentationInstance SegmentationObject DetectionObject RecognitionQuality MetricsSegmentation AnalysisAccuracyEvaluationEvaluation MetricsInstance Segmentation EvaluationMetricsQuality Gap AnalysisSegmentationSegmentation Quality Metrics
Designing metrics for evaluating instance segmentation revolves around comprehensively considering object detection and segmentation accuracy. However, other important properties, such as sensitivity, continuity, and equality, are overlooked in the current study. In this paper, we reveal that most existing metrics have a limited resolution of segmentation quality. They are only conditionally sensitive to the change of masks or false predictions. For certain metrics, the score can change drastically in a narrow range which could provide a misleading indication of the quality gap between results. Therefore, we propose a new metric called sortedAP, which strictly decreases with both object- and pixel-level imperfections and has an uninterrupted penalization scale over the entire domain. We provide the evaluation toolkit and experiment code at https://www.github.com/looooongChen/sortedAP.
13
Kaiming He, Georgia Gkioxari, Piotr Dollár et al. · 2017 · 28.8K citations
The Pascal Visual Object Classes (VOC) Challenge
Mark Everingham, Luc Van Gool, Christopher K. I. Williams et al. · International Journal of Computer Vision · 2009 · 19.5K citations · Full text
The Hungarian method for the assignment problem
Harold W. Kuhn · Naval Research Logistics Quarterly · 1955 · 12.6K citations
Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl
Juan C. Caicedo, Allen Goodman, Kyle W. Karhohs et al. · Nature Methods · 2019 · 853 citations · Full text