Semi-supervised learning using Gaussian fields and harmonic functions

TLDR

The learning algorithms are closely related to random walks, electric networks, and spectral graph theory. The paper proposes a semi‑supervised learning approach based on a Gaussian random field model. The method models data as a weighted graph, treats learning as a Gaussian random field whose mean is given by harmonic functions, and solves it efficiently with matrix techniques or belief propagation, while also incorporating class priors, supervised predictions, entropy‑based parameter learning, and feature selection. Experiments on synthetic, digit, and text data show promising results.

Abstract

An approach to semi-supervised learning is proposed that is based on a Gaussian random field model. Labeled and unlabeled data are represented as vertices in a weighted graph, with edge weights encoding the similarity between instances. The learning problem is then formulated in terms of a Gaussian random field on this graph, where the mean of the field is characterized in terms of harmonic functions, and is efficiently obtained using matrix methods or belief propagation. The resulting learning algorithms have intimate connections with random walks, electric networks, and spectral graph theory. We discuss methods to incorporate class priors and the predictions of classifiers obtained by supervised learning. We also propose a method of parameter learning by entropy minimization, and show the algorithm's ability to perform feature selection. Promising experimental results are presented for synthetic data, digit classification, and text classification tasks.