2022 · 24 citations · 22 references
ComputingComputer ScienceAcousticsFeature EngineeringMachine LearningDeep LearningPattern RecognitionRepresentation LearningTime Series Feature ExtractionAttentionAttention-based ClassificationAudio ClassificationAudio Feature ExtractionFrequency-domain FeaturesTime-frequency FeaturesTransformer ModelsTransformer-based Classification
Audio classification is an important task in the machine learning field with a wide range of applications. Since the last decade, deep learning based methods have been widely used and the transformer-based models are becoming new paradigm for audio classification. In this paper, we present Spectrogram Transformers, which are a group of transformer-based models for audio classification. Based on the fundamental semantics of audio spectrogram, we design two mechanisms to extract temporal and frequency features from audio spectrogram, named time-dimension sampling and frequency-dimension sampling. These discriminative representations are then enhanced by various combinations of attention block architectures, including Tempo-ral Only (TO) attention, Temporal-Frequency sequential (TFS) attention, Temporal-Frequency Parallel (TFP) attention, and Two-stream Temporal-Frequency (TSTF) attention, to extract the sound record signatures to serve the classification task. Our experiments demonstrate that these Transformer models outper-form the state-of-the-art methods on ESC-50 dataset without pre-training stage. Furthermore, our method also shows great efficiency compared with other leading methods.
22
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Daniel Park, William Chan, Yu Zhang et al. · 2019 · 3.5K citations · Full text
CNN architectures for large-scale audio classification
Shawn Hershey, Sourish Chaudhuri, Daniel P. W. Ellis et al. · 2017 · 2.5K citations
AST: Audio Spectrogram Transformer
Yuan Gong, Yu-An Chung, James Glass · 2021 · 989 citations