BMC Bioinformatics · 2023 · 41 citations · 31 references
Life SciencesComputer ScienceMicrobiologyBiomedical Text MiningMicrobiome ResearchNatural Language ProcessingGut MicrobiomeHost-microbiome InteractionsMicrobiome Data AnalysisMicrobiome DysbiosisMicrobiome FunctionInformation Extraction From Structured DocumentsMicrobe InteractionsRelation Extraction From Biomedical TextStructured Data Extraction
BACKGROUND: The growing recognition of the microbiome's impact on human health and well-being has prompted extensive research into discovering the links between microbiome dysbiosis and disease (healthy) states. However, this valuable information is scattered in unstructured form within biomedical literature. The structured extraction and qualification of microbe-disease interactions are important. In parallel, recent advancements in deep-learning-based natural language processing algorithms have revolutionized language-related tasks such as ours. This study aims to leverage state-of-the-art deep-learning language models to extract microbe-disease relationships from biomedical literature. RESULTS: In this study, we first evaluate multiple pre-trained large language models within a zero-shot or few-shot learning context. In this setting, the models performed poorly out of the box, emphasizing the need for domain-specific fine-tuning of these language models. Subsequently, we fine-tune multiple language models (specifically, GPT-3, BioGPT, BioMedLM, BERT, BioMegatron, PubMedBERT, BioClinicalBERT, and BioLinkBERT) using labeled training data and evaluate their performance. Our experimental results demonstrate the state-of-the-art performance of these fine-tuned models ( specifically GPT-3, BioMedLM, and BioLinkBERT), achieving an average F1 score, precision, and recall of over [Formula: see text] compared to the previous best of 0.74. CONCLUSION: Overall, this study establishes that pre-trained language models excel as transfer learners when fine-tuned with domain and problem-specific data, enabling them to achieve state-of-the-art results even with limited training data for extracting microbiome-disease interactions from scientific publications.
31
Advances in neural information processing systems 7
Krzysztof J. Cios, Mark E. Shields · Neurocomputing · 1997 · 22.3K citations
Re-epithelialization and immune cell behaviour in an ex vivo human skin model
Ana Rakita, Nenad Nikolić, Michael Mildner et al. · Scientific Reports · 2020 · 8.3K citations · Full text
Transformers: State-of-the-Art Natural Language Processing
Thomas Wolf, Lysandre Debut, Victor Sanh et al. · 2020 · 8K citations · Full text