An Evaluation of the Feasibility, Validity, and Reliability of Laparoscopic Skills Assessment in the Operating Room

TLDR

The study aims to evaluate a synchronized video‑based motion‑tracking device for instant, objective, automated assessment of laparoscopic skill in the operating room, and to demonstrate its validity, reliability, and feasibility. Nineteen surgeons (13 experienced, 6 inexperienced) performed 53 laparoscopic cholecystectomies on patients with biliary colic, with each procedure recorded by the ROVIMAS motion‑tracking system and subsequently rated by two observers using a validated operative assessment scale. Motion‑tracking metrics differed significantly between experienced and inexperienced surgeons during Calot triangle dissection, and the device showed strong inter‑test reliability and moderate to strong correlation (r = 0.4–0.7) with observer ratings, indicating an instant, valid, and reliable assessment that could streamline audit and credentialing.

Abstract

To assess the use of a synchronized video-based motion tracking device for objective, instant, and automated assessment of laparoscopic skill in the operating room.The assessment of technical skills is fundamental to recognition of proficient surgical practice. It is necessary to demonstrate the validity, reliability, and feasibility of any tool to be applied for objective measurement of performance.Nineteen subjects, divided into 13 experienced (performed >100 laparoscopic cholecystectomies) and 6 inexperienced (performed <10 LCs) surgeons completed LCs on 53 patients who all had a diagnosis of biliary colic. Each procedure was recorded with the ROVIMAS video-based motion tracking device to provide an objective measure of the surgeon's dexterity. Each video was also rated by 2 experienced observers on a previously validated operative assessment scale.There were significant differences for motion tracking parameters between the 2 groups of surgeons for the Calot triangle dissection part of procedure for time taken (P = 0.002), total path length (P = 0.026), and number of movements (P = 0.005). Both motion tracking and video-based assessment displayed intertest reliability, and there were good correlations between the 2 modes of assessment (r = 0.4 to 0.7, P < 0.01).An instant, objective, valid, and reliable mode of assessment of laparoscopic performance in the operating room has been defined. This may serve to reduce the time taken for technical skills assessment, and subsequently lead to accurate and efficient audit and credentialing of surgeons for independent practice.