The Art versus Science of Predicting Prognosis: Can a Prognostic Index Predict Short-Term Mortality Better than Experienced Nurses Do?

Abstract

OBJECTIVE: To determine whether a prognostic index could predict one-week mortality more accurately than hospice nurses can. METHOD: An electronic health record-based retrospective cohort study of 21,074 hospice patients was conducted in three hospice programs in the Southeast, Northeast, and Midwest United States. Model development used logistic regression with bootstrapped confidence intervals and multiple imputation to account for missing data. The main outcome measure was mortality within 7 days of hospice enrollment. RESULTS: A total of 21,074 patients were admitted to hospice between October 1, 2008 and May 31, 2011, and 5562 (26.4%) died within 7 days. An optimal predictive model included the Palliative Performance Scale (PPS) score, admission from a hospital, and gender. The model had a c-statistic of 0.86 in the training sample and 0.84 in the validation sample, which was greater than that of nurses' predictions (0.72). The index's performance was best for patients with pulmonary disease (0.89) and worst for patients with cancer and dementia (both 0.80). The index's predictions of mortality rates in each index category were within 5.0% of actual rates, whereas nurses underestimated mortality by up to 18.9%. Using the optimal index threshold (<3), the index's predictions had a better c-statistic (0.78 versus 0.72) and higher sensitivity (74.4% versus 47.8%) than did nurses' predictions but a lower specificity (80.6% versus 95.1%). CONCLUSIONS: Although nurses can often identify patients who will die within 7 days, a simple model based on available clinical information offers improved accuracy and could help to identify those patients who are at high risk for short-term mortality.