Robust Privacy-Preserving Recommendation Systems Driven by Multimodal Federated Learning

Abstract

Recommendation system (RS) is an important information filtering tool in nowadays digital era. With the growing concern on privacy, deploying RSs in a federated learning (FL) manner emerges as a promising solution, which can train a high-quality model on the premise that the server does not directly access sensitive user data. Nevertheless, some malicious clients can deduce user data by analyzing the uploaded model parameters. Even worse, some Byzantine clients can also send contaminated data to the server, causing blockage or failure of model convergence. In addition, most existing researches on federated recommendation algorithms only focus on unimodality learning, ignoring the assistance of multiple modality data to promote recommendation accuracy. Therefore, this article designs an FL-based privacy-preserving multimodal RS framework. To distinguish various modality data, an attention mechanism is introduced, wherein different weight ratios are assigned to various modal features. To further strengthen the privacy, local differential privacy (LDP) and personalized FL strategies are designed to identify malicious clients and bolster the resilience against Byzantine attacks. Finally, two multimodal datasets are established to verify the effectiveness of the proposed algorithm. The superiority of our proposed techniques is confirmed by the simulation results.