Use of Natural Language Processing Algorithms to Identify Common Data Elements in Operative Notes for Total Hip Arthroplasty

Abstract

Update This article was updated on December 6, 2019, because of a previous error. On page 1936, in Table VII, “Performance of the Bearing Surface Algorithm,” the row that had read “Bearing surface predicted by algorithm” now reads “Bearing surface predicted by algorithm*.” An erratum has been published: J Bone Joint Surg Am. 2020 Jan 2;102(1):e4. Update This article was updated on March 31, 2020, because of a previous error. On page 1934, in Table IV (“THA Bearing Surface-Related Keywords in Operative Notes”), the row that had read “Femoral stem; stem; HFx-stem; femoral component; femoral component/stem; permanent prosthesis; stem fem cemented” now reads “Femoral head; ball; delta head; delta ceramic head; ion treated; BIOLOX delta; ceramic head; ceramic femoral head; ceramic offset head; ceramic (size) head; alumina ceramic head; alumina prosthetic head; alumna ceramic head; BIOLOX ceramic head; BIOLOX delta head; BIOLOX femoral head; BIOLOX delta ceramic head.” An erratum has been published: J Bone Joint Surg Am. 2020 May 6;102(9):e43. Background: Manual chart review is labor-intensive and requires specialized knowledge possessed by highly trained medical professionals. Natural language processing (NLP) tools are distinctive in their ability to extract critical information from raw text in electronic health records (EHRs). As a proof of concept for the potential application of this technology, we examined the ability of NLP to correctly identify common elements described by surgeons in operative notes for total hip arthroplasty (THA). Methods: We evaluated primary THAs that had been performed at a single academic institution from 2000 to 2015. A training sample of operative reports was randomly selected to develop prototype NLP algorithms, and additional operative reports were randomly selected as the test sample. Three separate algorithms were created with rules aimed at capturing (1) the operative approach, (2) the fixation method, and (3) the bearing surface category. The algorithms were applied to operative notes to evaluate the language used by 29 different surgeons at our center and were applied to EHR data from outside facilities to determine external validity. Accuracy statistics were calculated with use of manual chart review as the gold standard. Results: The operative approach algorithm demonstrated an accuracy of 99.2% (95% confidence interval [CI], 97.1% to 99.9%). The fixation technique algorithm demonstrated an accuracy of 90.7% (95% CI, 86.8% to 93.8%). The bearing surface algorithm demonstrated an accuracy of 95.8% (95% CI, 92.7% to 97.8%). Additionally, the NLP algorithms applied to operative reports from other institutions yielded comparable performance, demonstrating external validity. Conclusions: NLP-enabled algorithms are a promising alternative to the current gold standard of manual chart review for identifying common data elements from orthopaedic operative notes. The present study provides a proof of concept for use of NLP techniques in clinical research studies and registry-development endeavors to reliably extract data of interest in an expeditious and cost-effective manner.