Multi-Source Medical Image Fusion Based on Wasserstein Generative Adversarial Networks

Abstract

In this paper, we propose the medical Wasserstein generative adversarial networks (MWGAN), an end-to-end model, for fusing magnetic resonance imaging (MRI) and positron emission tomography (PET) medical images. Our method establishes two adversarial games between a generator and two discriminators to generate a fused image with the details of soft tissue structures in organs from MRI images and the functional and metabolic information from PET images. Different information from source images can be effectively adjusted with a specifically designed loss function. In addition, we use WGAN instead of the traditional generative adversarial networks to make the training process more stable and allow our architecture to deal with source images of different resolutions. Qualitative and quantitative comparisons on publicly available datasets demonstrate the superiority of MWGAN over the state-of-the-art networks. Furthermore, our MWGAN is applied to the fusion of MRI and computed tomography images of different resolutions, achieving a satisfactory performance.