PET-MRI Sequence Fusion using Convolution Neural Network

Abstract

Combining positron emission tomography (PET) with magnetic resonance imaging (MRI) yields information that is complimentary from both a functional and anatomical standpoint. However, owing to the disparities in imaging physics and acquisition techniques, the integration of different modalities continues to be a difficult endeavor is challenge. Within the scope of this research, a deep learning-based strategy is presented in this study for PET-MRI sequence fusion that makes use of convolutional neural networks (CNNs). The proposed approach trains a CNN model to discover a mapping between the two modalities by capitalizing on the similarities that exist between the spatial and temporal characteristics of the two sequences. The proposed technique was tested using a dataset consisting of fifty PET-MRI scans. The findings illustrate the ability of our method to properly fuse the two sequences and increase picture quality in comparison to registration-based approaches that have been used traditionally. The CNN-based fusion strategy offers promise for enabling the clinical integration of PET-MRI, which would ultimately result in more accurate diagnosis and treatment planning for a variety of disorders.