2023 · 36 citations · 47 references
ComputingComputer ScienceComputer VisionMachine LearningMachine VisionDeep LearningGenerative ModelingImage ProcessingPattern RecognitionRepresentation LearningVideo AnalysisAutoencodersClassification AccuracyDiffusion ModelsImage InpaintingImage RepresentationInitializationInpaintingVideo Representation Learning
There has been a longstanding belief that generation can facilitate a true understanding of visual data. In line with this, we revisit generatively pre-training visual representations in light of recent interest in denoising diffusion models. While directly pre-training with diffusion models does not produce strong representations, we condition diffusion models on masked input and formulate diffusion models as masked autoencoders (DiffMAE). Our approach is capable of (i) serving as a strong initialization for downstream recognition tasks, (ii) conducting high-quality image inpainting, and (iii) being effortlessly extended to video where it produces state-of-the-art classification accuracy. We further perform a comprehensive study on the pros and cons of design choices and build connections between diffusion models and masked autoencoders. Project page.
47
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Kaiming He, Georgia Gkioxari, Piotr Dollár et al. · 2017 · 28.8K citations
A Fast Learning Algorithm for Deep Belief Nets
Geoffrey E. Hinton, Simon Osindero, Yee‐Whye Teh · Neural Computation · 2006 · 16.4K citations