What if your patient switches from Dr. Google to Dr. ChatGPT? A vignette-based survey of the trustworthiness, value, and danger of ChatGPT-generated responses to health questions

Abstract

ChatGPT is a new artificial intelligence system that revolutionizes the way how information can be sought and obtained. In this study, the trustworthiness, value, and danger of ChatGPT-generated responses on four vignettes that represented virtual patient questions were evaluated by 20 experts in the domain of congenital heart disease, atrial fibrillation, heart failure, or cholesterol. Experts generally considered ChatGPT-generated responses trustworthy and valuable, with few considering them dangerous. Forty percent of the experts found ChatGPT responses more valuable than Google. Experts appreciated the sophistication and nuances in the responses but also recognized that responses were often incomplete and sometimes misleading.