Plastic & Reconstructive Surgery Global Open · 2024 · 15 citations · 25 references
EducationPlastic SurgerySurgeryArtificial IntelligenceArtificial Intelligence In EducationNatural Language ProcessingArtificial Intelligence AdoptionConversational User InterfacesEducational ChatbotsNatural Language Processing In EducationAccuracyAi Performance MeasurementArtificial Intelligence EducationChatbot SystemsChatbotsIncorrect Response AnalysisSurgical Skill AssessmentSurgical Training
Background: Within the last few years, artificial intelligence (AI) chatbots have sparked fascination for their potential as an educational tool. Although it has been documented that one such chatbot, ChatGPT, is capable of performing at a moderate level on plastic surgery examinations and has the capacity to become a beneficial educational tool, the potential of other chatbots remains unexplored. Methods: To investigate the efficacy of AI chatbots in plastic surgery education, performance on the 2019-2023 Plastic Surgery In-service Training Examination (PSITE) was compared among seven popular AI platforms: ChatGPT-3.5, ChatGPT-4.0, Google Bard, Google PaLM, Microsoft Bing AI, Claude, and My AI by Snapchat. Answers were evaluated for accuracy and incorrect responses were characterized by question category and error type. Results: ChatGPT-4.0 outperformed the other platforms, reaching accuracy rates up to 79%. On the 2023 PSITE, ChatGPT-4.0 ranked in the 95th percentile of first-year residents; however, relative performance worsened when compared with upper-level residents, with the platform ranking in the 12th percentile of sixth-year residents. The performance among other chatbots was comparable, with their average PSITE score (2019-2023) ranging from 48.6% to 57.0%. Conclusions: Results of our study indicate that ChatGPT-4.0 has potential as an educational tool in the field of plastic surgery; however, given their poor performance on the PSITE, the use of other chatbots should be cautioned against at this time. To our knowledge, this is the first article comparing the performance of multiple AI chatbots within the realm of plastic surgery education.
25
Aidan Gilson, Conrad Safranek, Thomas Huang et al. · JMIR Medical Education · 2023 · 2K citations · Full text
Sai Anirudh Athaluri, Sandeep Varma Manthena, V S R Krishna Manoj Kesapragada et al. · Cureus · 2023 · 375 citations · Full text