International Journal of Computer Assisted Radiology and Surgery · 2023 · 25 citations · 22 references
ComputingComputer ScienceSurgeryBiomedical ImagingComputer VisionMachine LearningDeep LearningEndoscopic SurgeryImage SynthesisMedical Image AnalysisMedical ImagingVirtual EndoscopyComputer Vision For SurgeryEndoscopic SimulationEndoscopic VisionMedical Image SynthesisSynthetic Clinical DataSynthetic Medical Imaging DataThree-dimensional Endoscopic Imaging
PURPOSE: Computer-assisted surgical systems provide support information to the surgeon, which can improve the execution and overall outcome of the procedure. These systems are based on deep learning models that are trained on complex and challenging-to-annotate data. Generating synthetic data can overcome these limitations, but it is necessary to reduce the domain gap between real and synthetic data. METHODS: We propose a method for image-to-image translation based on a Stable Diffusion model, which generates realistic images starting from synthetic data. Compared to previous works, the proposed method is better suited for clinical application as it requires a much smaller amount of input data and allows finer control over the generation of details by introducing different variants of supporting control networks. RESULTS: The proposed method is applied in the context of laparoscopic cholecystectomy, using synthetic and real data from public datasets. It achieves a mean Intersection over Union of 69.76%, significantly improving the baseline results (69.76 vs. 42.21%). CONCLUSIONS: The proposed method for translating synthetic images into images with realistic characteristics will enable the training of deep learning methods that can generalize optimally to real-world contexts, thereby improving computer-assisted intervention guidance systems.
22
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Adding Conditional Control to Text-to-Image Diffusion Models
Lvmin Zhang, Anyi Rao, Maneesh Agrawala · 2023 · 3.6K citations
Holistically-Nested Edge Detection
Saining Xie, Zhuowen Tu · 2015 · 3.1K citations
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
Nataniel Ruiz, Yuanzhen Li, Varun Jampani et al. · 2023 · 1.9K citations