Safe Reinforcement Learning for Sepsis Treatment

Abstract

Sepsis, a life-threatening illness, is estimated to be the primary cause of death for 50,000 people a year in the UK and many more worldwide. Managing the treatment of sepsis is very challenging as it is frequently missed at an early stage and the optimal treatment is not yet clear. There are promising attempts to use Reinforcement Learning (RL) to learn optimal strategies to treat sepsis patients, especially for the administration of intravenous fluids and vasopressors. However, some RL agents only take the current state of patients into account when recommending the dosage of vasopressors. This is inconsistent with clinical safety practice in which the dosage of vasopressors is increased or decreased gradually. A sudden major change of the dosage might cause significant harm to patients and as such is considered unsafe in sepsis treatment. In this paper, we have adapted one of the deep RL methods published previously and evaluated whether the learned policy contains these sudden, major changes when recommending the vasopressor dosage. Then, we have modified this method to address the above safety constraint and learnt a safer policy by incorporating current clinical knowledge and practice.