Cross-modal Semantic Communications in 6G

Abstract

In 6G communication, semantic communication is considered one of the most promising directions to fulfill users’ demands for immersive multi-modal experiences, low latency, and high reliability. We proposes a cross-modal semantic communication approach based on deep learning, where both semantic coding and decoding are carefully crafted to provide optimum performance. Firstly, cross-modal semantic fusion is designed to enable end-to-end data transmission, driven by various task requirements of multi-modal business users. In addition, the proposed approach for evaluation on the semantic similarity is highly effective. It consists of a siamese network and a pseudo-siamese network, which can accurately obtain the matching loss between modal contents. Finally, the simulation results show that the proposed cross-modal semantic communication approach outperforms traditional communication systems, especially in low SNR scenarios. The similarity of cross-modal semantic communication improves by more than 53% compared to the traditional approaches, demonstrating its superiority and feasibility. Overall, our solution can meet the increasing demands of modern communication and facilitate seamless and intuitive experiences for users.