Neural Computation · 2022 · 149 citations · 57 references
ComputingComputer ScienceComputer VisionMachine LearningNatural Language ProcessingDeep LearningNeural NetworksNeural RepresentationRepresentation LearningContrastive LearningHierarchical ProcessingHierarchical Representation LearningImage RepresentationNeural Network ModelingNeural Representation LearningRepresentation Learning TheoryTransformer Language Models
This article does not describe a working system. Instead, it presents a single idea about representation that allows advances made by several different groups to be combined into an imaginary system called GLOM.1 The advances include transformers, neural fields, contrastive representation learning, distillation, and capsules. GLOM answers the question: How can a neural network with a fixed architecture parse an image into a part-whole hierarchy that has a different structure for each image? The idea is simply to use islands of identical vectors to represent the nodes in the parse tree. If GLOM can be made to work, it should significantly improve the interpretability of the representations produced by transformer-like systems when applied to vision or language.
57
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov et al. · arXiv (Cornell University) · 2020 · 21.6K citations · Full text
Advances in Neural Information Processing Systems 28
Peter Flach, Meelis Kull · 2015 · 14.7K citations
Advances in neural information processing systems 7
Computers & Mathematics with Applications · 1996 · 14.4K citations
Robust Real-Time Face Detection
Paul Viola, Michael Jones · International Journal of Computer Vision · 2004 · 14.2K citations