Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification

TLDR

Convolutional Neural Networks excel at image classification, yet training them on gigapixel Whole Slide Tissue Images is computationally infeasible, and cancer subtype differentiation relies on cellular‑level features visible only at the patch scale. The study aims to show that a patch‑level classifier can match or surpass an image‑level classifier and to devise a method for intelligently aggregating patch predictions while accounting for non‑discriminative patches. They train a decision‑fusion model that aggregates patch‑level CNN predictions and introduce an EM‑based approach that automatically identifies discriminative patches by exploiting spatial relationships among patches. The method achieves classification accuracy comparable to pathologist inter‑observer agreement and, on a smaller non‑cancer dataset, demonstrates that a patch‑based CNN can outperform an image‑based CNN.

Abstract

Convolutional Neural Networks (CNN) are state-of-theart models for many image classification tasks. However, to recognize cancer subtypes automatically, training a CNN on gigapixel resolution Whole Slide Tissue Images (WSI) is currently computationally impossible. The differentiation of cancer subtypes is based on cellular-level visual features observed on image patch scale. Therefore, we argue that in this situation, training a patch-level classifier on image patches will perform better than or similar to an image-level classifier. The challenge becomes how to intelligently combine patch-level classification results and model the fact that not all patches will be discriminative. We propose to train a decision fusion model to aggregate patch-level predictions given by patch-level CNNs, which to the best of our knowledge has not been shown before. Furthermore, we formulate a novel Expectation-Maximization (EM) based method that automatically locates discriminative patches robustly by utilizing the spatial relationships of patches. We apply our method to the classification of glioma and non-small-cell lung carcinoma cases into subtypes. The classification accuracy of our method is similar to the inter-observer agreement between pathologists. Although it is impossible to train CNNs on WSIs, we experimentally demonstrate using a comparable non-cancer dataset of smaller images that a patch-based CNN can outperform an image-based CNN.