Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis

TLDR

Overlap metrics such as Dice and Tanimoto are widely used to evaluate registration and segmentation, yet most studies report only a single average per region even when multiple labels and images are involved. This work generalizes common overlap measures to compute the total overlap of ensembles of labels across multiple test images and to incorporate fractional labels through fuzzy set theory. The authors introduce complementary overlap distance, related to the Hausdorff distance, and validate the generalized measures on synthetic images and in nonrigid registration of 3D MRI brain data. The resulting framework produces a single figure‑of‑merit that summarizes complex experiments, and it is applied to evaluate publicly available brain segmentation algorithms using a pragmatic atlas‑based ground truth.

Abstract

Measures of overlap of labelled regions of images, such as the Dice and Tanimoto coefficients, have been extensively used to evaluate image registration and segmentation algorithms. Modern studies can include multiple labels defined on multiple images yet most evaluation schemes report one overlap per labelled region, simply averaged over multiple images. In this paper, common overlap measures are generalized to measure the total overlap of ensembles of labels defined on multiple test images and account for fractional labels using fuzzy set theory. This framework allows a single "figure-of-merit" to be reported which summarises the results of a complex experiment by image pair, by label or overall. A complementary measure of error, the overlap distance, is defined which captures the spatial extent of the nonoverlapping part and is related to the Hausdorff distance computed on grey level images. The generalized overlap measures are validated on synthetic images for which the overlap can be computed analytically and used as similarity measures in nonrigid registration of three-dimensional magnetic resonance imaging (MRI) brain images. Finally, a pragmatic segmentation ground truth is constructed by registering a magnetic resonance atlas brain to 20 individual scans, and used with the overlap measures to evaluate publicly available brain segmentation algorithms.