PLoS Computational Biology · 2019 · 133 citations · 33 references
Life SciencesBioinformaticsFeature EngineeringGenomicsMachine LearningDeep LearningMotif DiscoveryRegulatory GenomicsRepresentation LearningSequence Pattern RecognitionConvolutional Neural NetworksDeep Feature LearningHierarchical Representation LearningInterpretable Neural NetworksLocalist RepresentationMotif Representation LearningRegulatory SequencesSequence Representation Learning
Although convolutional neural networks (CNNs) have been applied to a variety of computational genomics problems, there remains a large gap in our understanding of how they build representations of regulatory genomic sequences. Here we perform systematic experiments on synthetic sequences to reveal how CNN architecture, specifically convolutional filter size and max-pooling, influences the extent that sequence motif representations are learned by first layer filters. We find that CNNs designed to foster hierarchical representation learning of sequence motifs-assembling partial features into whole features in deeper layers-tend to learn distributed representations, i.e. partial motifs. On the other hand, CNNs that are designed to limit the ability to hierarchically build sequence motif representations in deeper layers tend to learn more interpretable localist representations, i.e. whole motifs. We then validate that this representation learning principle established from synthetic sequences generalizes to in vivo sequences.
33
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations
Advances in neural information processing systems 7
Krzysztof J. Cios, Mark E. Shields · Neurocomputing · 1997 · 22.3K citations
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2015 · 18.8K citations