Neurocomputing · 2023 · 32 citations · 16 references
ComputingComputer ScienceData ScienceDementiaMachine LearningNatural Language ProcessingDeep LearningDementia PrognosisMultimodal CorporaMultimodal Corpus AnalysisMultimodal InteractionMultimodal PhenotypingAudio ClassificationConvolutional Neural NetworksMultimodal ClassificationMultimodal Feature RepresentationMultimodal Health PredictionMultimodal Machine LearningMultimodal Representation LearningText Classification
This paper proposes a multimodal deep learning architecture combining text and audio information to predict dementia, a disease which affects around 55 million people all over the world and makes them in some cases dependent people. The system was evaluated on the DementiaBank Pitt Corpus dataset, which includes audio recordings as well as their transcriptions for healthy people and people with dementia. Different models have been used and tested, including Convolutional Neural Networks (CNN) for audio classification, Transformers for text classification, and a combination of both in a multimodal ensemble. These models have been evaluated on a test set, obtaining the best results by using the text modality, achieving 90.36% accuracy on the task of detecting dementia. Additionally, an analysis of the corpus has been conducted for the sake of explainability, aiming to obtain more information about how the models generate their predictions and identify patterns in the data.
16
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Andrew Howard, Menglong Zhu, Bo Chen et al. · arXiv (Cornell University) · 2017 · 9.9K citations · Full text
CNN architectures for large-scale audio classification
Shawn Hershey, Sourish Chaudhuri, Daniel P. W. Ellis et al. · 2017 · 2.5K citations
VisualBERT: A Simple and Performant Baseline for Vision and Language
Liunian Harold Li, Mark Yatskar, Da Yin et al. · arXiv (Cornell University) · 2019 · 1.2K citations · Full text
The Natural History of Alzheimer's Disease
James T. Becker · Archives of Neurology · 1994 · 556 citations