bioRxiv (Cold Spring Harbor Laboratory) · 2023 · 238 citations · 42 references
Life SciencesBioinformaticsProtein Structure PredictionStructural BioinformaticsStructural BiologyProtein StructureProtein Structure ModelingLarge Language Models
A bstract Large-scale protein language models (PLMs), such as the ESM family, have achieved remarkable performance in various downstream tasks related to protein structure and function by undergoing unsupervised training on residue sequences. They have become essential tools for researchers and practitioners in biology. However, a limitation of vanilla PLMs is their lack of explicit consideration for protein structure information, which suggests the potential for further improvement. Motivated by this, we introduce the concept of a “ s tructure- a ware vocabulary” that integrates residue tokens with structure tokens. The structure tokens are derived by encoding the 3D structure of proteins using Foldseek. We then propose SaProt, a large-scale general-purpose PLM trained on an extensive dataset comprising approximately 40 million protein sequences and structures. Through extensive evaluation, our SaProt model surpasses well-established and renowned baselines across 10 significant downstream tasks, demonstrating its exceptional capacity and broad applicability. We have made the code 1 , pre-trained model, and all relevant materials available at https://github.com/westlake-repl/SaProt .
42
Highly accurate protein structure prediction with AlphaFold
John Jumper, Alexander Pritzel, Tim Green et al. · Nature · 2021 · 45.2K citations · Full text
ClinVar: improving access to variant interpretations and supporting evidence
Melissa Landrum, Jennifer M. Lee, Mark J. Benson et al. · Nucleic Acids Research · 2017 · 4.5K citations · Full text
Language Models are Few-Shot Learners
T. B. Brown, Benjamin Mann, Nick Ryder et al. · arXiv (Cornell University) · 2020 · 3K citations · Full text