2015 · 364 citations · 36 references
ComputingComputer SciencePsychologyComputer VisionMachine LearningMemoryDeep LearningImage Feature EngineeringMemory AssessmentObject RecognitionRepresentation LearningVisual MemoryCognitive SystemsConvolutional Neural NetworksDeep Feature LearningDeep Image FeaturesDeep Learning-based ClassificationMemorabilityRank CorrelationVisual Recognition Memory
Progress in estimating visual memorability has been limited by the small scale and lack of variety of benchmark data. Here, we introduce a novel experimental procedure to objectively measure human memory, allowing us to build LaMem, the largest annotated image memorability dataset to date (containing 60,000 images from diverse sources). Using Convolutional Neural Networks (CNNs), we show that fine-tuned deep features outperform all other features by a large margin, reaching a rank correlation of 0.64, near human consistency (0.68). Analysis of the responses of the high-level CNN layers shows which objects and regions are positively, and negatively, correlated with memorability, allowing us to create memorability maps for each image and provide a concrete method to perform image memorability manipulation. This work demonstrates that one can now robustly estimate the memorability of images from many different classes, positioning memorability and deep memorability features as prime candidates to estimate the utility of information for cognitive systems. Our model and data are available at: http://memorability.csail.mit.edu.
36
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, Trevor Darrell · 2015 · 36.9K citations
Histograms of Oriented Gradients for Human Detection
Navneet Dalal, Bill Triggs · 2005 · 31.9K citations · Full text
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Ross Girshick, Jeff Donahue, Trevor Darrell et al. · 2014 · 31.8K citations