A retrofit eye gaze tracker for the da Vinci and its integration in task execution using the da Vinci Research Kit

Abstract

The integration of eye-gaze tracking at the console of surgical robots has the potential to add both speed and functionality to the human-robot interface. In this paper, we present a novel eye gaze tracker that can be integrated as a simple retrofit to the da Vinci console. In particular, the eye tracker can be used with the da Vinci Research Kit (dVRK) to control the patient side manipulators. We present the eye-tracker design and calibration. First, a 2D calibration is carried out to estimate the gaze on the da Vinci's stereoscopic display, followed by a 3D “hand-eye” calibration to estimate the gaze in the surgical scene. Using the dVRK, we demonstrated the use of the eye-tracker to perform a gaze-assisted task, in which the estimated gaze is used as a set point for the robot and the user is guided towards the point of gaze through haptic feedback. The task performed was peg placement and was evaluated in a preliminary user study with five users.