Federated Learning for Image Semantic Communication System Based on CNN and Transformer

Abstract

The letter presents a semantic communication approach that combines Convolutional Neural Network (CNN) and Transformer models. By using CNN as a feature extractor to convert image features into feature sequences, which are then processed by the Transformer model, this method captures spatial relationships and contextual information in images, enabling better image reconstruction in image transmission tasks. Furthermore, the letter applies federated learning to the proposed model. Client devices train the model locally and then send the updated model parameters to a central server for aggregation, achieving global model training without centrally storing the original dataset on the server. Federated learning not only addresses data privacy protection concerns but also reduces the training time for each user. Experimental results demonstrate the effectiveness of combining CNN and Transformer models on different datasets, while federated learning significantly improves the model's performance and privacy protection capabilities. Therefore, the proposed method in the letter can serve as an effective image modality semantic communication encoder.