Journal of Cheminformatics · 2020 · 265 citations · 47 references
ComputingComputer ScienceMedicinal ChemistryDrug DesignDrug DiscoveryMachine LearningDeep LearningRepresentation LearningTransfer LearningBenchmark DatasetsConsensusDeep Learning For Molecular PropertiesDeep Learning-based ClassificationMolecular Representation LearningTransfer Learning For Molecular Property Prediction
We present SMILES-embeddings derived from the internal encoder state of a Transformer [1] model trained to canonize SMILES as a Seq2Seq problem. Using a CharNN [2] architecture upon the embeddings results in higher quality interpretable QSAR/QSPR models on diverse benchmark datasets including regression and classification tasks. The proposed Transformer-CNN method uses SMILES augmentation for training and inference, and thus the prognosis is based on an internal consensus. That both the augmentation and transfer learning are based on embeddings allows the method to provide good results for small datasets. We discuss the reasons for such effectiveness and draft future directions for the development of the method. The source code and the embeddings needed to train a QSAR model are available on https://github.com/bigchem/transformer-cnn. The repository also has a standalone program for QSAR prognosis which calculates individual atoms contributions, thus interpreting the model's result. OCHEM [3] environment (https://ochem.eu) hosts the on-line implementation of the method proposed.
47
Leo Breiman · Machine Learning · 2001 · 126.2K citations · Full text
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Chih-Chung Chang, Chih‐Jen Lin · ACM Transactions on Intelligent Systems and Technology · 2011 · 41.3K citations
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations