Recognition of Echolalic Autistic Child Vocalisations Utilising Convolutional Recurrent Neural Networks

Abstract

Autism spectrum conditions (ASC) are a set of neurodevelopmental conditions partly characterised by difficulties with communication.Individuals with ASC can show a variety of atypical speech behaviours, including echolalia or the 'echoing' of another's speech.We herein introduce a new dataset of 15 Serbian ASC children in a human-robot interaction scenario, annotated for the presence of echolalia amongst other ASC vocal behaviours.From this, we propose a four-class classification problem and investigate the suitability of applying a 2D convolutional neural network augmented with a recurrent neural network with bidirectional long short-term memory cells to solve the proposed task of echolalia recognition.In this approach, log Mel-spectrograms are first generated from the audio recordings and then fed as input into the convolutional layers to extract high-level spectral features.The subsequent recurrent layers are applied to learn the long-term temporal context from the obtained features.Finally, we use a feed forward neural network with softmax activation to classify the dataset.To evaluate the performance of our deep learning approach, we use leave-onesubject-out cross-validation.Key results presented indicate the suitability of our approach by achieving a classification accuracy of 83.5 % unweighted average recall.