Journal of Organizational and End User Computing · 2021 · 54 citations · 24 references
ComputingComputer ScienceMachine LearningNatural Language ProcessingText AnalyticsDeep LearningNatural Language UnderstandingRepresentation LearningSentiment AnalysisText Feature EngineeringWord EmbeddingsConvolutional Neural NetworksDeep Feature FusionFeature FusionSemantic Feature ModelingText ClassificationText Feature FusionTransformer Language ModelsWord Representation Learning
Short text classification is a research focus for natural language processing (NLP), which is widely used in news classification, sentiment analysis, mail filtering and other fields. In recent years, deep learning techniques are applied to text classification and has made some progress. Different from ordinary text classification, short text has the problem of less vocabulary and feature sparsity, which raise higher request for text semantic feature representation. To address this issue, this paper propose a feature fusion framework based on the Bidirectional Encoder Representations from Transformers (BERT). In this hybrid method, BERT is used to train word vector representation. Convolutional neural network (CNN) capture static features. As a supplement, a bi-gated recurrent neural network (BiGRU) is adopted to capture contextual features. Furthermore, an attention mechanism is introduced to assign the weight of salient words. The experimental results confirmed that the proposed model significantly outperforms the other state-of-the-art baseline methods.
24
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Distributed Representations of Words and Phrases and their Compositionality
Tomáš Mikolov, Ilya Sutskever, Kai Chen et al. · arXiv (Cornell University) · 2013 · 18.1K citations · Full text
Convolutional Neural Networks for Sentence Classification