Cell Genomics · 2025 · 10 citations · 43 references
Life SciencesBioinformaticsGeneticsGene ExpressionGenomicsTranscriptional RegulationFunctional GenomicsGene Regulatory ElementsRegulatory GenomicsCell Type-specific Regulatory ElementsCell-type-specific Regulatory ElementsChromatin AccessibilityCis-regulatory LogicEnhancer Prediction ModelsLearned Feature RepresentationsMulti-modal Representation LearningRegulatory Motifs
Sequence-based deep learning models have emerged as powerful tools for deciphering the cis-regulatory grammar of the human genome but cannot generalize to unobserved cellular contexts. Here, we present EpiBERT, a multi-modal transformer that learns generalizable representations of genomic sequence and cell type-specific chromatin accessibility through a masked accessibility-based pre-training objective. Following pre-training, EpiBERT can be fine-tuned for gene expression prediction, achieving accuracy comparable to the sequence-only Enformer model, while also being able to generalize to unobserved cell states. The learned representations are interpretable and useful for predicting chromatin accessibility quantitative trait loci (caQTLs), regulatory motifs, and enhancer-gene links. Our work represents a step toward improving the generalization of sequence-based deep neural networks in regulatory genomics.
43
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Model-based Analysis of ChIP-Seq (MACS)
Yong Zhang, Tao Liu, Clifford A. Meyer et al. · Genome biology · 2008 · 19.9K citations · Full text
NCBI GEO: archive for functional genomics data sets—update
Tanya Barrett, Stephen E. Wilhite, Pierre Ledoux et al. · Nucleic Acids Research · 2012 · 10.8K citations · Full text
Decoupled Weight Decay Regularization
Ilya Loshchilov, Frank Hutter · arXiv (Cornell University) · 2017 · 9.1K citations · Full text