Graph Representation Learning via Graphical Mutual Information Maximization

TLDR

Graphical data such as social and communication networks offer unprecedented potential for learning high‑quality representations without external supervision. The study aims to preserve and extract abundant information from graph‑structured data into an embedding space in an unsupervised manner. We introduce Graphical Mutual Information (GMI), a graph‑domain mutual information measure that captures correlations between node features and topology, and use it to train an unsupervised graph neural encoder by maximizing GMI between input graphs and their hidden representations. Experiments on node classification and link prediction show that the GMI‑based unsupervised model outperforms state‑of‑the‑art unsupervised methods and sometimes surpasses supervised baselines, while GMI is invariant to graph isomorphisms, efficiently estimable via MINE, and theoretically sound.

Abstract

The richness in the content of various information networks such as social networks and communication networks provides the unprecedented potential for learning high-quality expressive representations without external supervision. This paper investigates how to preserve and extract the abundant information from graph-structured data into embedding space in an unsupervised manner. To this end, we propose a novel concept, Graphical Mutual Information (GMI), to measure the correlation between input graphs and high-level hidden representations. GMI generalizes the idea of conventional mutual information computations from vector space to the graph domain where measuring mutual information from two aspects of node features and topological structure is indispensable. GMI exhibits several benefits: First, it is invariant to the isomorphic transformation of input graphs—an inevitable constraint in many existing graph representation learning algorithms; Besides, it can be efficiently estimated and maximized by current mutual information estimation methods such as MINE; Finally, our theoretical analysis confirms its correctness and rationality. With the aid of GMI, we develop an unsupervised learning model trained by maximizing GMI between the input and output of a graph neural encoder. Considerable experiments on transductive as well as inductive node classification and link prediction demonstrate that our method outperforms state-of-the-art unsupervised counterparts, and even sometimes exceeds the performance of supervised ones.