IEEE Transactions on Neural Networks · 1995 · 660 citations · 28 references
Data RepresentationAssociative MemoriesGeometric LearningEngineeringMachine LearningAutoencodersGeometric QuantizationRecurrent Neural NetworkData SciencePattern RecognitionFeature LearningQuantum Field TheoryComputer EngineeringComputer ScienceComputer VisionCircular ConvolutionRepresentation TheoryGroup RepresentationNoisy Reconstructions
Associative memories typically encode simple data as sets of vector pairs. The paper proposes a method to encode complex compositional structures in distributed representations. It employs circular convolution to bind items represented as vectors. The method enables arbitrary variable bindings, short sequences, and simple frame‑like structures in fixed‑width vectors, and noisy reconstructions can be cleaned with a separate associative memory.
Associative memories are conventionally used to represent data with very simple structure: sets of pairs of vectors. This paper describes a method for representing more complex compositional structure in distributed representations. The method uses circular convolution to associate items, which are represented by vectors. Arbitrary variable bindings, short sequences of various lengths, simple frame-like structures, and reduced representations can be represented in a fixed width vector. These representations are items in their own right and can be used in constructing compositional structures. The noisy reconstructions extracted from convolution memories can be cleaned up by using a separate associative memory that has good reconstructive properties.
28