Can donor narratives yield insights? A natural language processing proof of concept to facilitate kidney allocation

Abstract

Although expedited placement could ameliorate stagnant kidney utilization, precisely identifying difficult-to-place organs is crucial to mitigate potential harms associated with this policy. Existing algorithms have only leveraged structured data from the Organ Procurement and Transplantation Network (OPTN); however, detailed, free text case information about a donor exists. No known research exists about the utility of these data. We developed a model to predict the probability of delay or discard for adult deceased kidney donors between 2010 and 2018, leveraging donor free text data. The resultant model had a c-statistic of 0.75 compared to 0.80 (Reduced Probability of Delay or Discard [model], r-PODD) and 0.77 (Kidney Donor Profile Index, KDPI) on the test dataset. Analysis of the top predictive words suggest both known and potentially novel clinical factors (ie, a known factor such as hypertension vs a novel factor such as stents), and nuanced social factors (intravenous drug use) could negatively affect kidney utilization. These findings suggest that donor narratives have utility; the natural language processing (NLP) model is only moderately correlated with existing indices and provides directional evidence about additional cardiovascular risk factors that may affect kidney utilization. More research is needed to understand the potential to enhance existing indices of kidney utilization to better enable and mitigate the effects of policy interventions such as expedited placement. Although expedited placement could ameliorate stagnant kidney utilization, precisely identifying difficult-to-place organs is crucial to mitigate potential harms associated with this policy. Existing algorithms have only leveraged structured data from the Organ Procurement and Transplantation Network (OPTN); however, detailed, free text case information about a donor exists. No known research exists about the utility of these data. We developed a model to predict the probability of delay or discard for adult deceased kidney donors between 2010 and 2018, leveraging donor free text data. The resultant model had a c-statistic of 0.75 compared to 0.80 (Reduced Probability of Delay or Discard [model], r-PODD) and 0.77 (Kidney Donor Profile Index, KDPI) on the test dataset. Analysis of the top predictive words suggest both known and potentially novel clinical factors (ie, a known factor such as hypertension vs a novel factor such as stents), and nuanced social factors (intravenous drug use) could negatively affect kidney utilization. These findings suggest that donor narratives have utility; the natural language processing (NLP) model is only moderately correlated with existing indices and provides directional evidence about additional cardiovascular risk factors that may affect kidney utilization. More research is needed to understand the potential to enhance existing indices of kidney utilization to better enable and mitigate the effects of policy interventions such as expedited placement. Although 13 patients with end-stage renal disease (ESRD) die each day awaiting a transplant on the Kidney Waitlist,1UNOSOPTN Database [database online]. United Network for Organ Sharing, Richmond, VA2019Google Scholar 20% of recovered kidneys are not transplanted. Not only could the need for deceased donor kidneys increase in the near future,2University of MichiganKidney Epidemiology and Cost Center. Report for the percentage of prevalent patients waitlisted (PPPW). CMS, 2018Google Scholar the discard rate has stagnated between 18% and 20%.3Reese PP Harhay MN Abt PL Levine MH Halpern SD. New solutions to reduce discard of kidneys donated for transplantation.J Am Soc Nephrol. 2015; 27: 973-980Crossref PubMed Scopus (97) Google Scholar Although the static kidney discard rate has been associated with the proportion of marginal deceased donor kidneys within the donor pool,4Stewart DE Garcia VC Rosendale JD Klassen DK Carrico BJ. Diagnosing the decades-long rise in the deceased donor kidney discard rate in the United States.Transplantation. 2017; 101: 575-587Crossref PubMed Scopus (95) Google Scholar there is evidence to suggest that some of these kidneys are viable for transplantation.5Massie AB Luo X Chow EK Alejo JL Desai NM Segev DL. Survival benefit of primary deceased donor transplantation with high-KDPI kidneys.Am J Transplant. 2014; 14: 2310-2316Crossref PubMed Scopus (156) Google Scholar Therefore, there is a need to make sure that every viable kidney is placed with a suitable recipient. Not every candidate is medically suitable or willing to accept a marginal deceased donor kidney; however, the current kidney allocation policy requires organs to be offered to candidates in the order that they appear on the match run, without explicit consideration to the candidate’s or their center’s likelihood of acceptance. Although the Collaborative Innovation and Improvement Network (COIIN) has shown promise to improve kidney utilization within the existing allocation paradigm,6Sisaithong KN Carrico RJ Reddy KS. Preliminary findings from the first pilot phase of the collaborative innovation and improvement network.Transplantation. 2018; 102: 1590-1592Crossref PubMed Scopus (2) Google Scholar stagnant kidney utilization rates and highly variable candidate/center preferences have led to calls for developing policy interventions. A potential policy intervention, often dubbed Expedited Placement, could improve kidney utilization by allowing organs to be offered preferentially to candidates at centers with a history of transplanting similar, hard-to-place organs if the likelihood of organ discard (or delayed acceptance) exceeds a specified threshold.7Cooper M Formica R Friedewald J et al.Report of national kidney foundation consensus conference to decrease kidney discards.Clin Transplant. 2019; 33: e13419Crossref PubMed Scopus (43) Google Scholar Although this alternative allocation policy framework could improve system efficiency and increase the number of transplants, a top Organ Procurement and Transplantation Network (OPTN) strategic goal,8OPTN Strategic Plan 2018–2021. 2018. https://optn.transplant.hrsa.gov/governance/strategic-plan/. Accessed April 19, 2019.Google Scholar it has the potential to have an adverse effect on some individuals; some may not receive kidney offers under an expedited system that they would have received and accepted under the current system. Hence, a critical challenge for any alternative allocation policy is to minimize the potential negative impact on equity by identifying as accurately as possible the kidneys most likely to be discarded under the current system. Given the potential importance of precision in identifying likely-to-be-discarded organs within an alternative kidney allocation policy, researchers have developed sophisticated models to predict deceased donor kidneys that will experience placement difficulties.9Massie AB, Desai NM, Montgomery RA, Singer AL, Segev DL. Improving distribution efficiency of hard-to-place deceased donor kidneys: Predicting probability of discard or delay. Am J Transplant. 2010;1613-1620.Google Scholar, 10Zhou S Massie AB Holscher CM et al.Prospective validation of prediction model for kidney discard.Transplantation. 2019; 103: 764-771Crossref PubMed Scopus (11) Google Scholar, 11Marrero WJ Naik AS Friedewald JJ et al.Predictors of deceased donor kidney discard in the United States.Transplantation. 2017; 101: 1690-1697Crossref PubMed Scopus (41) Google Scholar Although these models have exhibited strong performance rank ordering, their sensitivity is limited.9Massie AB, Desai NM, Montgomery RA, Singer AL, Segev DL. Improving distribution efficiency of hard-to-place deceased donor kidneys: Predicting probability of discard or delay. Am J Transplant. 2010;1613-1620.Google Scholar, 10Zhou S Massie AB Holscher CM et al.Prospective validation of prediction model for kidney discard.Transplantation. 2019; 103: 764-771Crossref PubMed Scopus (11) Google Scholar, 11Marrero WJ Naik AS Friedewald JJ et al.Predictors of deceased donor kidney discard in the United States.Transplantation. 2017; 101: 1690-1697Crossref PubMed Scopus (41) Google Scholar The potential to improve these models is limited if further attempts are based solely on the same traditional OPTN data sources. Often the only way to improve model performance is by incorporating new data sources or finding new insights within the data.12Domingos P. A few useful things to know about machine learning.Commun ACM. 2012; 55: 78Crossref Scopus (1567) Google Scholar One previously overlooked data source is “unstructured” data, or the narratives entered by the Organ Procurement Organization (OPO) to communicate to transplant hospitals information about the donor’s admission course, medical and social history, and other “highlights” relevant to donation (collectively “donor text”). These data represent a new opportunity for exploration. There has been little, if any, published research on the utility of text fields entered by OPOs. OPTN policies, specifically 2.11, may warrant more investigation of free text narratives. OPOs are required to provide social and behavioral histories.13OPTN. Policies Section 2.11. 2019. https://optn.transplant.hrsa.gov/media/1200/optn_policies.pdf. Accessed June 14, 2019.Google Scholar Many OPTN fields are limited to Yes/No questions, which may not provide sufficient detail to enable fully informed organ offer acceptance. This constraint is not new within the broader field of health care; “workarounds” via free text fields have been documented in the medical informatics literature.14Collins SA Fred M Wilcox L Vawdrey DK. Workarounds used by nurses to overcome design constraints of electronic health records.NI. 2012; 2012: 93Google Scholar The theoretical framework of workarounds, found in many healthcare environments, states that individuals find means to address and overcome inadequate process and information flows.15Alter S. Theory of workarounds.Comm Assoc Inf Syst. 2014; 34 (https://doi.org/10.17705/1CAIS.03455)Google Scholar Donor text may fall into the workaround paradigm by allowing users to communicate necessary information, such as detailed US Public Health Service (“PHS”) increased risk reasons that are not captured in sufficient detail elsewhere, in addition to detailed donor admission history. We posit that donor text is sufficient to predict kidney utilization and that natural language processing (NLP) could reveal nuanced clinical text that may be associated with kidney utilization. Institutional review board (IRB) exemption for this study was obtained from Chesapeake IRB (now Advarra). This study used data from the OPTN, whose data system includes data on all donors, waitlisted candidates, and transplant recipients in the United States, submitted by the members of the OPTN (https://optn.transplant.hrsa.gov/data/about-data/). Although the structured data, Deceased Donor Record (DDR), Transplant Candidate Record (TCR), and Transplant Recipient Record (TRR) have been described elsewhere,16Massie AB Kucirka LM Segev DL. Big data in organ transplantation: registries and administrative claims.Am J Transplant. 2014; 14: 1723-1730Crossref PubMed Scopus (219) Google Scholar the unstructured data pertaining to donor records in UNOS databases have not been previously characterized. Unstructured donor data used in this study, the aforementioned “donor text,” comes from three free text fields available in DonorNet. The fields are known as admission course history, donor medical and social history, and donor highlights. To our knowledge, all three of these fields are entered by OPOs, usually from the on-site coordinator or via deceased donor records, as part of the donor assessment/procurement process. Although DonorNet users are free to input any relevant information in each of the free text boxes, we assessed each of these fields. Donor Highlights provide details on important clinical, logistical, and potentially unusual circumstances pertaining to the donor in question.17UNOS. Donor Highlights Resource. UNOS Connect. https://unosconnect.unos.org/RadEditorData/Site1/resources/sys119/donor_highlights.pdf. Accessed June 20, 2019.Google Scholar According to the UNOS Department of Professional Education, specific reasons for PHS increased risk may be found in donor highlights in addition to issues with organ function and notes about other organs that may be ruled out already.17UNOS. Donor Highlights Resource. UNOS Connect. https://unosconnect.unos.org/RadEditorData/Site1/resources/sys119/donor_highlights.pdf. Accessed June 20, 2019.Google Scholar These data are not captured elsewhere by the OPTN. Hence, Donor Highlights often contain “need to know” information about the donor. Admission course describes the clinical situation that led to the patient becoming an organ donor. Often these narratives describe “how” a person became a donor. Typically the data are a text blurb. An illustrative example follows:“patient was found in room with needles nearby. Downtime unknown. Naloxone administered on scene” Donor medical and social history describes the donor’s relevant pyschosocial and medical history. Our understanding is that data within these histories may be perceived to lead to a higher risk of disease contraction or other deleterious effects on the transplant suitability of organs. Another illustrative example follows:“historian mentioned ivda user (last use unknown); drank daily.” Here≥ we review the used to and our prediction model with the A J in J 2019; PubMed Scopus Google Scholar This process is in with and to the and of the We the data from adult of deceased kidney donors recovered between 2010 and 2018. We only donors had at kidney recovered for the of deceased donor kidney we to our study at the donor to the kidney and not kidneys to specific such as or candidates or evidence that of all kidney within the DE Klassen the of deceased donor kidney allocation in the 2018; Google Scholar the study to allocation policy may our to deceased donor kidneys at risk of Our for a donor was that found in such as and are not available at the of There is evidence from the medical informatics that of consideration for the of data at the of prediction in prediction performance to leveraging data model S clinical data for a 2018; Google Scholar we found that for a of the placement are for an of donors both kidneys are or had only had kidney and the other We our of as delay or Our of a discard or delay is as at of the kidneys was discarded on more of or had an expedited the at their as under current allocation policy, placed a kidney out of with a known to accept such kidneys in order to organ We donor narratives by the admission course history, donor medical and social history, and donor highlights text fields from DonorNet. These be we the most the first kidney match as and not to the donor The text into DonorNet was first to reduce To and words from all in a process text by to their or and are into of Scholar and a of words both by the in Accessed 19, 2019.Google Scholar The donor into a of donor the to Scholar The of this framework is to words based on important they are to each which could be associated with our of J A of Scopus Google Scholar without to order such that text of has and and has and would be into Although more sophisticated that into are they more data and that are the of this We leveraged the of in A et machine in Scholar the for each at We our data into and was the performance and the additional described on the test This that the to text We a framework to predict a kidney would have is an the number of is compared to the number of text R The of and New Google Scholar We the in the for the and We via as described in the We the to to for the are more or We leveraged a validation with the to models with and to we each models and on of the The that the c-statistic the was and the was used to a model with the We the described with this model on the test used to model performance predictive and The c-statistic and We leveraged the the of Scholar to within the a model and a donor text the a of words from the by the words affect the model prediction and in which (ie, transplant or We to every donor text in the test To a of the most important words for our for each we the top words that transplant and the top words that and the all to many a was as We to understand the between our donor text model and other indices in order to understand if we are the same in a way known as in new the of this study, we to our model with the S Massie AB Holscher CM et al.Prospective validation of prediction model for kidney discard.Transplantation. 2019; 103: 764-771Crossref PubMed Scopus (11) Google Scholar known as and Our for and was that these models only likely to be available at of the match S Massie AB Holscher CM et al.Prospective validation of prediction model for kidney discard.Transplantation. 2019; 103: 764-771Crossref PubMed Scopus (11) Google Scholar To the indices to each we model at the donor We and to understand the and performance of each provides a understanding of each for of each in of sensitivity and review for J PubMed Scopus Google Scholar Our adult deceased donors into and test with at kidney for the of between 2010 and 2018. to structured data, we from the and from the test There donors, or of the with at kidney delayed or our of difficult-to-place we have a the of each of to of of donor with kidneys with of and expedited in the match as to in of this donor with at kidney discarded on donor organ and expedited in the match as to in of this donor with a kidney placed with more of and expedited in the match as to in of this donor with at kidney placed within of and an expedited in the match of as to in of this donor with a kidney that exhibited a of discard the of our variable in addition to each to the in of number of donors and of as to in of this in a new This the of our variable in addition to each to the in of number of donors and of The validation via the for the donor model was 0.75 between and The test data c-statistic was Not only is there a of performance in each a performance on a that the model was not the model our or delay or and the same on the data, the c-statistic was 0.80 to our test dataset. The model in a test c-statistic of Hence, the based on a donor to be better at rank the the model only of all or at the the and Although the models sensitivity at the there is only between the model and other The and the model have a of and and the model have a of that the model may not be leveraging the same for of and To understand the to difficult-to-place kidney donors, via we a predictive and by for each model with a higher of test performance in of by model on the test each in of and on the test dataset. in a new by This that all three and predictive performance with a higher predictive performance be at This each in of and on the test dataset. The words that most important to as by the the of Scholar are shown in such as and many words discard or transplant predictive words a of clinical as as social and and from these drug use be from some transplant as as a donor with an discard or delay often clinical some of which may be on clinical in DonorNet An of cardiovascular factors shown to be associated with discard if they not on the of the top predictive words (ie, A of and their be found in the some to kidney transplantation or transplantation in strong predictive of these in the donor text suggest that they appear in or on the part of for to renal the model may relevant information from transplant The of novel known risk and predictive performance this model has relevant for transplant or Our that free text donor narratives could be a data source that could our current prediction of kidney utilization. we found that an based solely on free text donor narratives a of predictive as traditional indices developed structured data The OPTN such as and within the structured the model Our model that nuanced PHS risk data cardiovascular risk and some other clinical factors (ie, may affect kidney utilization, which are not captured in structured data by the OPTN. Although our to DonorNet narratives is to useful information from free text fields is not new to transplantation or health in pilot within the field of transplantation have the of to improve the prediction of et data, predictive and improvement in kidney transplantation: a of J Transplant. 2017; PubMed Scopus Google A S of natural language processing in J Transplant. 2017; PubMed Scopus Google Scholar There of use of free text and in health S. language processing of documented in narratives of electronic health a Am 2019; PubMed Scopus Google S L A Big data in health to and and 2014; 33: PubMed Scopus Google Scholar to be an on the Although this model may be perceived as a we have leveraged a to provide the into words within the free text are the the of Scholar the model highlights specific of PHS risk (ie, drug use) associated with This finding is with research on kidney offer Holscher CM S et for associated with increased risk kidneys.Am J Transplant. 2018; PubMed Scopus Google Scholar in addition to the and use used in AB, Desai NM, Montgomery RA, Singer AL, Segev DL. Improving distribution efficiency of hard-to-place deceased donor kidneys: Predicting probability of discard or delay. Am J Transplant. 2010;1613-1620.Google Scholar, 10Zhou S Massie AB Holscher CM et al.Prospective validation of prediction model for kidney discard.Transplantation. 2019; 103: 764-771Crossref PubMed Scopus (11) Google Scholar, 11Marrero WJ Naik AS Friedewald JJ et al.Predictors of deceased donor kidney discard in the United States.Transplantation. 2017; 101: 1690-1697Crossref PubMed Scopus (41) Google Scholar a directional understanding of these nuanced in to PHS is important that the for and is PHS risk of and of the for Health of the Department of Health and for to the PHS for and Organ 2019. 2019.Google Scholar Given that the model with is some of the more novel factors by the The model a number of cardiovascular factors that have not been previously words such as and negatively affect kidney utilization within this This finding if the on kidney disease and cardiovascular S The between kidney disease and cardiovascular 2014; Google Scholar in of the that kidney disease is J et kidney disease and US to Am Soc Nephrol. PubMed Scopus Google Scholar These findings could be on the of additional data to or to better understand and kidney utilization. Although we have that the on free text donor narratives we that more to To existing models requires developing from text to to if we improve existing utilization and Although we have not a to structured and unstructured data into a a of is as more sources of unstructured data are has shown promise to from A J in J 2019; PubMed Scopus Google Scholar into the the field of transplantation may benefit from an to identifying data that is not in the structured OPTN data fields. existing text fields in the OPTN such as from an or may be to improve of organ utilization. the with more data we further our existing utilization models and to understand is the The use of this model in a specific not further in this study, we DonorNet users know that their donor text will be by a may to words or associated with their (ie, they to the to the model is to the words or not will it may be possible to text that from is entered One as in the is the between efficiency and Given that the of this was to sources of data to ameliorate this we that in the model was to a sensitivity on however, the for each model was to that of all performance was more which the in identifying organs that will experience Hence, to organs may need to or not to a a delay to alternative allocation policy based on the number of The would to the UNOS research for their The of this have of to as described by the of The data that the findings of this study are available via from the OPTN. information may be found in the with