Journal of Pathology Informatics · 2025 · 16 citations · 10 references
ComputingBiomedical InformaticsComputer ScienceBiomedical Text MiningClinical InformaticsNatural Language ProcessingAnnotationClinical Natural Language ProcessingData ExtractionInformation ExtractionKnowledge ExtractionBiomedical Information ExtractionBreast Cancer HistopathologyClinical Text AnnotationInformation Extraction From Biomedical TextInformation Extraction From Structured DocumentsPathology Report ProcessingPathology ReportsStructured Data Extraction
Background: Structured information extraction from unstructured histopathology reports facilitates data accessibility for clinical research. Manual extraction by experts is time-consuming and expensive, limiting scalability. Large language models (LLMs) offer efficient automated extraction through zero-shot prompting, requiring only natural language instructions without labeled data or training. We evaluate LLMs' accuracy in extracting structured information from breast cancer histopathology reports, compared to manual extraction by a trained human annotator. Methods: We developed the Medical Report Information Extractor, a web application leveraging LLMs for automated extraction. We also developed a gold-standard extraction dataset to evaluate the human annotator alongside five LLMs including GPT-4o, a leading proprietary model, and the Llama 3 model family, which allows self-hosting for data privacy. Our assessment involved 111 breast cancer histopathology reports from the Generations study, extracting 51 pathology features specified within the study's data dictionary. Results: < 0.001), its reduced computational requirements make it a viable option for self-hosting. Conclusion: We developed an open-source tool for structured information extraction that demonstrated expert human-level accuracy in our evaluation using state-of-the-art LLMs. The tool can be customized by non-programmers using natural language and the modular design enables reuse for diverse extraction tasks to produce standardized, structured data facilitating analytics through improved accessibility and interoperability.
10
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent et al. · Foundations and Trends® in Machine Learning · 2020 · 4.7K citations · Full text
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli et al. · arXiv (Cornell University) · 2021 · 2.2K citations · Full text
A critical assessment of using ChatGPT for extracting structured data from clinical notes
Jingwei Huang, Donghan M. Yang, Ruichen Rong et al. · npj Digital Medicine · 2024 · 242 citations · Full text