2014 · 93 citations · 42 references
ComputingComputer ScienceComputational LinguisticsKnowledge RepresentationNatural Language ProcessingCompositionalityLanguage ModelingLanguage ModelsNatural Language UnderstandingParsingWord EmbeddingsCo-occurrence ModelingCompositional ModelingDialogue Act AnalysisNeural Text RepresentationParaphrase GenerationParaphrase RecognitionSentence SimilarityWord Representation Learning
We provide a comparative study between neural word representations and traditional vector spaces based on cooccurrence counts, in a number of compositional tasks. We use three different semantic spaces and implement seven tensor-based compositional models, which we then test (together with simpler additive and multiplicative approaches) in tasks involving verb disambiguation and sentence similarity. To check their scalability, we additionally evaluate the spaces using simple compositional methods on larger-scale tasks with less constrained language: paraphrase detection and dialogue act tagging. In the more constrained tasks, co-occurrence vectors are competitive, although choice of compositional method is important; on the largerscale tasks, they are outperformed by neural word embeddings, which show robust, stable performance across the tasks.
42
Distributed Representations of Words and Phrases and their Compositionality
Tomáš Mikolov, Ilya Sutskever, Kai Chen et al. · arXiv (Cornell University) · 2013 · 18.1K citations · Full text
Efficient Estimation of Word Representations in Vector Space
Tomáš Mikolov, Kai Chen, Greg S. Corrado et al. · arXiv (Cornell University) · 2013 · 11.7K citations · Full text
A unified architecture for natural language processing
Ronan Collobert, Jason Weston · 2008 · 5.2K citations