American Journal of Otolaryngology · 2025 · 13 citations · 5 references
MedicineOtolaryngologySurgeryClinical Decision SupportDiagnostic ReasoningOtolaryngologic SurgeryClinical ReasoningEar SurgeryLaryngeal SurgeryArtificial Intelligence-assisted DiagnosisClinical Diagnostic ReasoningClinical Reasoning SupportClinical Relevance AssessmentDiagnostic Accuracy StudiesDiagnostic Testing AccuracyKnowledge Representation In MedicineSurgery Decision-making
PURPOSE: This study aimed to evaluate the diagnostic accuracy, comprehensiveness, and clinical relevance of two advanced artificial intelligence (AI) models, OpenAI's ChatGPT-4.0 and DeepSeek-R1, in the field of otolaryngology. METHODS: Five common otolaryngology procedures-adenotonsillectomy, tympanoplasty, endoscopic sinus surgery, parotidectomy, and total laryngectomy-were analyzed through standardized queries posed to both AI models. Because the prompts replicate questions that patients typically search online, our evaluation focuses on patient-facing informational adequacy. Responses were independently evaluated by two study members for accuracy, clinical relevance, and comprehensiveness, with discrepancies resolved through consensus. The analysis included comparison with clinical guidelines. RESULTS: ChatGPT-4.0 generally provided detailed procedural insights, effectively covering indications, methodologies, risks, and recovery processes. However, it occasionally suggested excessive diagnostic imaging and omitted subtle yet significant surgical nuances. DeepSeek-R1 delivered concise, structured responses clearly categorizing indications, treatment alternatives, and procedural risks. Nonetheless, it frequently lacked detailed elaboration, omitting important surgical techniques and minor complications. For instance, DeepSeek-R1 omitted specifics such as hemostatic techniques in adenotonsillectomy and graft stabilization details in tympanoplasty. Neither model adequately addressed critical elements like comprehensive staging, detailed surgical planning, and long-term recovery nuances, especially for complex procedures such as total laryngectomy. CONCLUSIONS: Both ChatGPT-4.0 and DeepSeek-R1 demonstrated significant diagnostic potential but revealed limitations in precision, comprehensiveness, and nuanced clinical reasoning. Their clinical utility remains restricted, highlighting a continued need for AI refinement to enhance patient-specific decision-making capabilities in otolaryngology.
5