Fake News Detection via Multi-scale Semantic Alignment and Cross-modal Attention

Abstract

The effective utilization of multimodal information is crucial for the task of fake news detection. The primary challenge lies in reducing the semantic distance between the news evidences of different modalities and accurately aligning them in the latent space. It has been observed that the relevance of news image content varies across coarse-to-fine scales, depending on the object and its class label. For instance, the visual evidence to justify a news item sometimes may be a very local detail such as the tiny changes of facial expression and limb position while sometimes may be the global composition. Consequently, how to align news text with image at the most discriminative scale significantly impacts detection performance. However, very few studies have addressed this issue in fake news detection. In this paper, we delve deeper into this issue and propose a simple yet effective Multi-scale Semantic Alignment and Cross-modal Attention (MSACA) network. Specifically, we construct hierarchical multi-scale images for each news data, enhance the semantic consistency between text and images in the latent space, and employ an attention module to select the deterministic embeddings in an end-to-end manner. Extensive experiments on two real-world benchmarks demonstrate the superior performance of our proposed MSACA network.