medRxiv · 2023 · 24 citations · 6 references
ComputingBiomedical InformaticsHealth InformaticsClinical InformaticsElectronic Health RecordsNatural Language ProcessingClinical Data ModelingClinical DocumentationClinical Natural Language ProcessingElectronic Health Record ChartingHealth Information RetrievalInformation ExtractionClinical Language ModelsCommon Data ElementsElectronic Health Record DataElectronic Health Record Data AnalysisElectronic Health Record Data ExtractionLarge Language ModelsMedical Chart Review
Importance: Large language models (LLMs) have proven useful for extracting data from publicly available sources, but their uses in clinical settings and with clinical data are unknown. Objective: To determine the accuracy of data extraction using "Versa Chat," a chat implementation of the general-purpose OpenAI gpt-35-turbo LLM model, versus manual chart review for hepatocellular carcinoma (HCC) imaging reports. Design: We engineered a prompt for the data extraction task of six distinct data elements and input 182 abdominal imaging reports that were also manually tagged. We evaluated performance by calculating accuracy, precision, recall, and F1 scores. Setting/Participants: Cross-sectional abdominal imaging reports of patients diagnosed with hepatocellular carcinoma enrolled in the Functional Assessment in Liver Transplantation (FrAILT) study.
6
Large language models encode clinical knowledge
Karan Singhal, Shekoofeh Azizi, Tao Tu et al. · Nature · 2023 · 3.2K citations · Full text
Computing, Health Informatics, Computational Linguistics +16
What Makes Good In-Context Examples for GPT-3?
Jiachang Liu, Dinghan Shen, Yizhe Zhang et al. · 2022 · 497 citations · Full text
Prompt Engineering for Healthcare: Methodologies and Applications
Jiaqi Wang, Enze Shi, Sigang Yu et al. · arXiv (Cornell University) · 2023 · 118 citations · Full text