SAIL: A Deep-Learning-Based System for Automatic Gait Assessment From TUG Videos

Abstract

Gait disorders are common in the elderly people, seriously hinder patients&#x2019; mobility and sometimes indicate underlying severe neurological diseases. Timely and automatic diagnosis of gait disorders is greatly desired. Existing methods with wearable devices put burdens on patients. We establish a video-based algorithm named <i>SAIL</i> to perform contactless gait assessment automatically. The SAIL contains three parts, namely, <i>skeleton detector</i>, <i>parameter extractor</i>, and <i>gait classifier</i>. Using a pose estimation algorithm, the skeleton detector converts RGB videos to a human skeleton sequence. Then, the parameter extractor extracts gait parameters from skeletons with a signal detection technique. Finally, a trained Support vector machine is used as a gait classifier to detect abnormal gait. The SAIL achieves 86.2&#x0025; sensitivity and 98.5&#x0025; specificity for abnormal gait detection on our <i>SAIL-TUG</i> dataset, outperforming general clinic doctors with 76.4&#x0025; and 97.4&#x0025;, respectively. Nine gait parameters and the binary gait classification result are included in the final gait report. We implement an automatic gait assessment system based on SAIL and deployed the user-interface software in more than 60 hospitals for practical applications. More than 30 000 gait reports have been automatically generated. Moreover, we establish a publicly available dataset named <i>SAIL-TUG</i> including 404 annotated Timed &#x201C;Up &amp; Go&#x201D; videos.