Validation Study of a Virtual Reality Robotic Simulator—Role as an Assessment Tool?

TLDR

Virtual‑reality simulators are widely used for surgical skill training, but rigorous construct and criterion validity must be demonstrated before they can serve as assessment tools. The study aimed to evaluate the face, content, construct, and concurrent validity of the dV‑Trainer robotic surgical simulator. Urology residents, fellows, and attending surgeons performed three repetitions of four virtual‑reality tasks on the dV‑Trainer and, one week later, similar tasks on the da Vinci® robot, with performance measured by total task time and errors. Results showed that 77 % of novices and 71 % of experienced surgeons rated the dV‑Trainer as realistic or useful, experienced surgeons outperformed novices in many tasks, and task time and error scores on the dV‑Trainer correlated significantly with those on the da Vinci® robot, confirming its validity as an assessment tool.

Abstract

Virtual reality simulators are often used for surgical skill training since they facilitate deliberate practice in a controlled, low stakes environment. However, to be considered for assessment purposes rigorous construct and criterion validity must be demonstrated. We performed face, content, construct and concurrent validity testing of the dV-Trainer™ robotic surgical simulator.Urology residents, fellows and attending surgeons were enrolled in this institutional review board approved study. After a brief introduction to the dV-Trainer each subject completed 3 repetitions each of 4 virtual reality tasks on it, including pegboard ring transfer, matchboard object transfer, needle threading of rings, and the ring and rail task. One week later subjects completed 4 similar tasks using the da Vinci® robot. Subjects were assessed on total task time and total errors using the built-in scoring algorithm and manual scoring for the dV-Trainer and the da Vinci robot, respectively.Seven experienced and 13 novice robotic surgeons were included in the study. Experienced surgeons were defined by greater than 50 hours of clinical robotic console time. Of novice robotic surgeons 77% ranked the dV-Trainer as a realistic training platform and 71% of experienced robotic surgeons ranked it as useful for resident training. Experienced robotic surgeons outperformed novices in many dV-Trainer and da Vinci robot exercises, particularly in the number of errors. On pooled data analysis dV-Trainer total task time and total errors correlated with da Vinci robot total task time and total errors (p = 0.026 and 0.011, respectively).This study confirms the face, content, construct and concurrent validity of the dV-Trainer, which may have a potential role as an assessment tool.