Orthopedics · 2023 · 46 citations · 10 references
MedicineOrthopaedic SurgeryOrthopedic SurgeryArtificial IntelligenceClinical Decision SupportOrthopedicsOrthopedic ExaminationSurgery EducationSurgical ResidencyAcademic ProficiencyAutomated Performance MetricsOrthopedic Physical ExaminationSurgical Skill AssessmentSurgical Skill TrainingSurgical Training
Advances in artificial intelligence and machine learning models, like Chat Generative Pre-trained Transformer (ChatGPT), have occurred at a remarkably fast rate. OpenAI released its newest model of ChatGPT, GPT-4, in March 2023. It offers a wide range of medical applications. The model has demonstrated notable proficiency on many medical board examinations. This study sought to assess GPT-4's performance on the Orthopaedic In-Training Examination (OITE) used to prepare residents for the American Board of Orthopaedic Surgery (ABOS) Part I Examination. The data gathered from GPT-4's performance were additionally compared with the data of the previous iteration of ChatGPT, GPT-3.5, which was released 4 months before GPT-4. GPT-4 correctly answered 251 of the 396 attempted questions (63.4%), whereas GPT-3.5 correctly answered 46.3% of 410 attempted questions. GPT-4 was significantly more accurate than GPT-3.5 on orthopedic board-style questions ( P <.00001). GPT-4's performance is most comparable to that of an average third-year orthopedic surgery resident, while GPT-3.5 performed below an average orthopedic intern. GPT-4's overall accuracy was just below the approximate threshold that indicates a likely pass on the ABOS Part I Examination. Our results demonstrate significant improvements in OpenAI's newest model, GPT-4. Future studies should assess potential clinical applications as AI models continue to be trained on larger data sets and offer more capabilities. [ Orthopedics . 2024;47(2):e85–e89.]
10
Aidan Gilson, Conrad Safranek, Thomas Huang et al. · JMIR Medical Education · 2023 · 2K citations · Full text
Capabilities of GPT-4 on Medical Challenge Problems
Harsha Nori, Nicholas King, Scott Mayer McKinney et al. · arXiv (Cornell University) · 2023 · 510 citations · Full text
Performance of ChatGPT and GPT-4 on Neurosurgery Written Board Examinations
Rohaid Ali, Oliver Y. Tang, Ian D. Connolly et al. · Neurosurgery · 2023 · 209 citations