Nature Communications · 2024 · 53 citations · 27 references
MedicineComputer ScienceRadiologyComputer VisionDiagnostic ImagingDiagnostic RadiologyChest RadiographyChest X-ray InterpretationDeep LearningMedical ImagingRadiographic ImagingRepresentation LearningChest X-ray Pattern RecognitionComputer-aided Chest X-ray DiagnosisContrastive LearningDeep Learning In RadiologyFine-grained Image ClassificationMedical Image FusionReport Generation From Chest X-raysZero-shot Classification
Recently, multi-modal vision-language foundation models have gained significant attention in the medical field. While these models offer great opportunities, they still face crucial challenges, such as the requirement for fine-grained knowledge understanding in computer-aided diagnosis and the capability of utilizing very limited or even no task-specific labeled data in real-world clinical applications. In this study, we present MaCo, a masked contrastive chest X-ray foundation model that tackles these challenges. MaCo explores masked contrastive learning to simultaneously achieve fine-grained image understanding and zero-shot learning for a variety of medical imaging tasks. It designs a correlation weighting mechanism to adjust the correlation between masked chest X-ray image patches and their corresponding reports, thereby enhancing the model's representation learning capabilities. To evaluate the performance of MaCo, we conducted extensive experiments using 6 well-known open-source X-ray datasets. The experimental results demonstrate the superiority of MaCo over 10 state-of-the-art approaches across tasks such as classification, segmentation, detection, and phrase grounding. These findings highlight the significant potential of MaCo in advancing a wide range of medical image analysis tasks.
27
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford, Jong Wook Kim, Chris Hallacy et al. · arXiv (Cornell University) · 2021 · 5.3K citations · Full text
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Jeremy Irvin, Pranav Rajpurkar, Michael Ko et al. · 2019 · 2.6K citations · Full text