Exploring the use of ChatGPT in predicting anterior circulation stroke functional outcomes after mechanical thrombectomy: a pilot study

Abstract

BACKGROUND: Accurate prediction of functional outcomes is crucial in stroke management, but this remains challenging. OBJECTIVE: To evaluate the performance of the generative language model ChatGPT in predicting the functional outcome of patients with acute ischemic stroke (AIS) 3 months after mechanical thrombectomy (MT) in order to assess whether ChatGPT can used to be accurately predict the modified Rankin Scale (mRS) score at 3 months post-thrombectomy. METHODS: We conducted a retrospective analysis of clinical, neuroimaging, and procedure-related data from 163 patients with AIS undergoing MT. The agreement between ChatGPT's exact and dichotomized predictions and actual mRS scores was assessed using Cohen's κ. The added value of ChatGPT was measured by evaluating the agreement of predicted dichotomized outcomes using an existing validated score, the MT-DRAGON. RESULTS: ChatGPT demonstrated fair (κ=0.354, 95% CI 0.260 to 0.448) and good (κ=0.727, 95% CI 0.620 to 0.833) agreement with the true exact and dichotomized mRS scores at 3 months, respectively, outperforming MT-DRAGON in overall and subgroup predictions. ChatGPT agreement was higher for patients with shorter last-time-seen-well-to-door delay, distal occlusions, and better modified Thrombolysis in Cerebral Infarction scores. CONCLUSIONS: ChatGPT adequately predicted short-term functional outcomes in post-thrombectomy patients with AIS and was better than the existing risk score. Integrating AI models into clinical practice holds promise for patient care, yet refining these models is crucial for enhanced accuracy in stroke management.