Concepedia

Learning task-dependent distributed representations by backpropagation through structure

Proceedings of International Conference on Neural Networks (ICNN'96) · 2002 · 592 citations · 9 references

TL;DR

Neural networks excel at processing fixed-length vectors and variable-length sequences, but current methods cannot efficiently handle structured objects of arbitrary shape such as logical terms, trees, or graphs. The authors introduce a connectionist architecture with a novel supervised learning scheme capable of solving inductive inference tasks on complex symbolic structures of arbitrary size. Their method processes labeled directed acyclic graphs, tunes structure representations specifically for the inference task, and applies this to classifying logical terms—including detecting subterms and satisfying unification patterns. The approach achieved superior results compared to previous methods in classifying logical terms.

Abstract

While neural networks are very successfully applied to the processing of fixed-length vectors and variable-length sequences, the current state of the art does not allow the efficient processing of structured objects of arbitrary shape (like logical terms, trees or graphs). We present a connectionist architecture together with a novel supervised learning scheme which is capable of solving inductive inference tasks on complex symbolic structures of arbitrary size. The most general structures that can be handled are labeled directed acyclic graphs. The major difference of our approach compared to others is that the structure-representations are exclusively tuned for the intended inference task. Our method is applied to tasks consisting in the classification of logical terms. These range from the detection of a certain subterm to the satisfaction of a specific unification pattern. Compared to previously known approaches we obtained superior results in that domain.