How Segment Anything Model (Sam) Boost Medical Image Segmentation: A Survey

TLDR

Foundation models, especially the prompt‑driven Segment Anything Model (SAM), have revolutionized NLP and image generation, but their applicability to medical image segmentation remains uncertain due to fundamental differences between natural and medical imagery. This survey reviews recent efforts to adapt SAM for medical image segmentation, covering empirical benchmarking, methodological modifications, and future research directions. The authors compile a continuously updated paper list and open‑source project summary at https://github.com/YichiZhang98/SAM4MIS to facilitate research on SAM in medical image segmentation. Direct application of SAM to multi‑modal, multi‑target medical datasets yields unsatisfactory performance, yet the survey extracts insights that can guide future development of foundation models for medical image analysis.

Abstract

Due to the flexibility of prompting, foundation models have become the dominant force in the domains of natural language processing and image generation. With the recent introduction of the Segment Anything Model (SAM), the prompt-driven paradigm has entered the realm of image segmentation, bringing with a range of previously unexplored capabilities. However, it remains unclear whether it can be applicable to medical image segmentation due to the significant differences between natural images and medical images. In this work, we summarize recent efforts to extend the success of SAM to medical image segmentation tasks, including both empirical benchmarking and methodological adaptations, and discuss potential future directions for SAM in medical image segmentation. Although directly applying SAM to medical image segmentation cannot obtain satisfying performance on multi-modal and multi-target medical datasets, many insights are drawn to guide future research to develop foundation models for medical image analysis. We also set up a continuously updated paper list and open-source project summary to boost the research on this topic at https://github.com/YichiZhang98/SAM4MIS.