Radiology Artificial Intelligence · 2022 · 21 citations · 11 references
ComputingHealth InformaticsRadiologyClinical InformaticsDiagnostic RadiologyNatural Language ProcessingClinical Natural Language ProcessingDiagnostic ErrorDiagnostic Error DetectionRadiology InformaticsRadiology Information SystemsSpeech RecognitionRadiology ErrorsRadiology Natural Language ProcessingRadiology Report Interpretation ErrorsRadiology Report ProcessingRadiology ReportingRecognition ErrorsSpeech Recognition EvaluationSpeech Recognition In Radiology
Purpose: To develop radiology domain-specific bidirectional encoder representations from transformers (BERT) models that can identify speech recognition (SR) errors and suggest corrections in radiology reports. Materials and Methods: = 92). Correction Radiology BERT was separately trained to suggest corrections for detected deletion and substitution errors. Area under the receiver operating characteristic curve (AUC) and bootstrapped 95% CIs were calculated for each evaluation dataset. Results: Radiology-specific BERT had AUC values of >.99 (95% CI: >0.99, >0.99), 0.94 (95% CI: 0.93, 0.94), 0.98 (95% CI: 0.98, 0.98), and 0.97 (95% CI: 0.97, 0.97) for detecting insertion, deletion, substitution, and all errors, respectively, on the independently generated test set. Testing on unaltered report impressions revealed a sensitivity of 82% (28 of 34; 95% CI: 70%, 93%) and specificity of 88% (1521 of 1728; 95% CI: 87%, 90%). Testing on prospective SR errors showed an accuracy of 75% (69 of 92; 95% CI: 65%, 83%). Finally, the correct word was the top suggestion for 45.6% (475 of 1041; 95% CI: 42.5%, 49.3%) of errors. Conclusion: © RSNA, 2022See also the commentary by Abajian and Cheung in this issue.
11