arXiv (Cornell University) · 2022 · 31 citations · 0 references
ComputingComputer SciencePathologyComputer VisionDiagnostic ImagingMachine LearningCancer DetectionDeep LearningFeature DetectionMedical Image AnalysisMedical ImagingTumor DetectionConvolutional Neural Network DetectionConvolutional Neural NetworksDigital Pathology AnalysisInductive BiasPatch ClassificationSelf-supervised PretrainingTransformer ModelsVision Transformers
Vision transformers, with weaker inductive bias and more flexible feature detection, have shown the potential to outperform convolutional neural networks when pretrained on large datasets, though they may need more challenging tasks to realize this advantage. The study investigates vision transformers for tumor detection across four tissue types and for tissue type identification in digital pathology whole slide images. The authors compared patch‑wise classification of the DeiT‑Tiny vision transformer to the ResNet18 CNN, and also evaluated both models pretrained with self‑supervised learning on large unlabeled whole‑slide image datasets. Results show that vision transformers slightly outperformed ResNet18 on three of four tissue types for tumor detection, while ResNet18 performed better on the remaining tasks; aggregated slide‑level predictions were correlated, indicating similar feature capture, and overall both models performed comparably, though transformers demanded more training effort.
Recently, vision transformers were shown to be capable of outperforming convolutional neural networks when pretrained on sufficient amounts of data. In comparison to convolutional neural networks, vision transformers have a weaker inductive bias and therefore allow a more flexible feature detection. Due to their promising feature detection, this work explores vision transformers for tumor detection in digital pathology whole slide images in four tissue types, and for tissue type identification. We compared the patch-wise classification performance of the vision transformer DeiT-Tiny to the state-of-the-art convolutional neural network ResNet18. Due to the sparse availability of annotated whole slide images, we further compared both models pretrained on large amounts of unlabeled whole-slide images using state-of-the-art self-supervised approaches. The results show that the vision transformer performed slightly better than the ResNet18 for three of four tissue types for tumor detection while the ResNet18 performed slightly better for the remaining tasks. The aggregated predictions of both models on slide level were correlated, indicating that the models captured similar imaging features. All together, the vision transformer models performed on par with the ResNet18 while requiring more effort to train. In order to surpass the performance of convolutional neural networks, vision transformers might require more challenging tasks to benefit from their weak inductive bias.