2020 · 46 citations · 20 references
EngineeringRobotics EngineeringSurgeryHuman-robot InteractionMedical RoboticsRoboticsHuman-robot Interaction In HealthcarePolicy LearningReinforcement LearningRobot ManipulationRobot-assisted SurgerySurgical RoboticsHaptic FeedbackHuman-robot Collaboration In SurgeryInterface ControlRobot-assisted SuturingRobotic SuturingSparse Reward DesignSparse Reward LearningSurgical Manipulation
Over the past decade, Robot-Assisted Surgeries (RAS), have become more prevalent in facilitating successful operations. Of the various types of RAS, the domain of collaborative surgery has gained traction in medical research. Prominent examples include providing haptic feedback to sense tissue consistency, and automating sub-tasks during surgery such as cutting or needle hand-off - pulling and reorienting the needle after insertion during suturing. By fragmenting suturing into automated and manual tasks the surgeon could essentially control the process with one hand and also circumvent workspace restrictions imposed by the control interface present at the surgeon's side during the operation. This paper presents an exploration of a discrete reinforcement learning-based approach to automate the needle hand-off task. Users were asked to perform a simple running suture using the da Vinci Research Kit. The user trajectory was learnt by generating a sparse reward function and deriving an optimal policy using Q-learning. Trajectories obtained from three learnt policies were compared to the user defined trajectory. The results showed a root-mean-square error of [0.0044mm, 0.0027mm, 0.0020mm] in ℝ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> . Additional trajectories from varying initial positions were produced from a single policy to simulate repeated passes of the hand-off task.
20
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2016 · 6.8K citations · Full text
Computing, Computer Science, Physics +17
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · arXiv (Cornell University) · 2013 · 5.1K citations · Full text
Trust Region Policy Optimization
John Schulman, Sergey Levine, Philipp Moritz et al. · arXiv (Cornell University) · 2015 · 3.1K citations · Full text