DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning

TLDR

Large‑scale annotated radiological image datasets are crucial yet difficult to build; hospitals store many radiologist‑marked bookmarks in PACS that highlight significant findings across diverse lesion types such as lung nodules, liver tumors, and lymph nodes. The study aims to mine retrospective PACS bookmarks to construct a large‑scale lesion image dataset. The authors mined institutional PACS bookmarks to create DeepLesion—a 32,735‑lesion, 32,120‑slice dataset—and trained a unified detector to locate all lesion types with minimal manual effort. The resulting detector attains 81.1 % sensitivity at five false positives per image and offers broad applicability across medical imaging tasks.

Abstract

Extracting, harvesting, and building large-scale annotated radiological image datasets is a greatly important yet challenging problem. Meanwhile, vast amounts of clinical annotations have been collected and stored in hospitals' picture archiving and communication systems (PACS). These types of annotations, also known as bookmarks in PACS, are usually marked by radiologists during their daily workflow to highlight significant image findings that may serve as reference for later studies. We propose to mine and harvest these abundant retrospective medical data to build a large-scale lesion image dataset. Our process is scalable and requires minimum manual annotation effort. We mine bookmarks in our institute to develop DeepLesion, a dataset with 32,735 lesions in 32,120 CT slices from 10,594 studies of 4,427 unique patients. There are a variety of lesion types in this dataset, such as lung nodules, liver tumors, enlarged lymph nodes, and so on. It has the potential to be used in various medical image applications. Using DeepLesion, we train a universal lesion detector that can find all types of lesions with one unified framework. In this challenging task, the proposed lesion detector achieves a sensitivity of 81.1% with five false positives per image.