Graph Contrastive Learning with Augmentations

TLDR

Graph neural networks struggle to learn generalizable, transferable, and robust representations, and self‑supervised learning is underexplored compared to CNNs. The authors propose GraphCL, a graph contrastive learning framework for unsupervised representation learning on graph data. GraphCL employs four types of graph augmentations and evaluates their combinations across multiple datasets in semi‑supervised, unsupervised, transfer learning, and adversarial attack settings. GraphCL achieves comparable or superior generalizability, transferability, and robustness to state‑of‑the‑art methods without tuning augmentation extents or complex GNNs, and further gains are observed with parameterized augmentations; code is publicly available.

Abstract

Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training are less explored for GNNs. In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We first design four types of graph augmentations to incorporate various priors. We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. We also investigate the impact of parameterized graph augmentation extents and patterns, and observe further performance gains in preliminary experiments. Our codes are available at https://github.com/Shen-Lab/GraphCL.