Scientific Reports · 2021 · 41 citations · 38 references
MedicineBiomedical InformaticsPathologyBiomedical Text MiningClinical InformaticsElectronic Health RecordsAnnotationClinical Natural Language ProcessingHealth RecordsPatient IdentificationProtected Health InformationAnnotator AgreementClinical Text AnnotationDeviation ScoresInter-annotator AgreementInter-observer AgreementPathology ReportsPatient IdentifiersSensitive Information Protection
For research purposes, protected health information is often redacted from unstructured electronic health records to preserve patient privacy and confidentiality. The OpenDeID corpus is designed to assist development of automatic methods to redact sensitive information from unstructured electronic health records. We retrieved 4548 unstructured surgical pathology reports from four urban Australian hospitals. The corpus was developed by two annotators under three different experimental settings. The quality of the annotations was evaluated for each setting. Specifically, we employed serial annotations, parallel annotations, and pre-annotations. Our results suggest that the pre-annotations approach is not reliable in terms of quality when compared to the serial annotations but can drastically reduce annotation time. The OpenDeID corpus comprises 2,100 pathology reports from 1,833 cancer patients with an average of 737.49 tokens and 7.35 protected health information entities annotated per report. The overall inter annotator agreement and deviation scores are 0.9464 and 0.9726, respectively. Realistic surrogates are also generated to make the corpus suitable for distribution to other researchers.
38
MIMIC-III, a freely accessible critical care database
Alistair E. W. Johnson, Tom Pollard, Lu Shen et al. · Scientific Data · 2016 · 8.1K citations · Full text
Multiple significance tests: the Bonferroni method
Martin Bland, Doug Altman · BMJ · 1995 · 3.9K citations · Full text
Medicine, Immunology, Oncology +17
Inter-Coder Agreement for Computational Linguistics
Ron Artstein, Massimo Poesio · Computational Linguistics · 2008 · 1.6K citations · Full text