Mask <scp>RCNN</scp> algorithm for nuclei detection on breast cancer histopathological images

Abstract

Abstract Nuclei detection is a key step in computer assisted pathology. Due to the variability of the size, shape, appearance, and texture of breast cancer nuclei in histopathological images, automated nuclei detection has always been a difficult aspect of computer‐aided pathology research. In this article, Mask RCNN is presented for the automatic detection of nuclei on high‐resolution histopathological images of breast cancer. Mask RCNN uses the ResNet network and effectively combines modules such as feature pyramid networks (FPN), ROIAlign, and fully convolutional networks (FCN). FPN can efficiently extract features of various dimensions in images. ROIAlign can improve the accuracy of the detection model in the detection task. FCN renders the prediction results more detailed. The experiment results show that the application of this algorithm is superior to other algorithms in terms of its intuitive vision, as well as in performance indicators such as accuracy, recall, and F‐Measure.