Apollo (University of Cambridge) · 2018 · 547 citations · 160 references
ComputingComputer ScienceApplied LinguisticsComputational LinguisticsNatural Language ProcessingCross-linguistic TransferLanguage ModelingMachine TranslationMultilingual Natural Language ProcessingMultilingual ProcessingWord EmbeddingsCross-lingual Language ModelingCross-lingual Representation LearningCross-lingual TransferCross-lingual Transfer LearningCross-lingual Word EmbeddingsLanguage Processing ModelsMultilingual Word Embeddings
Cross‑lingual word embeddings enable reasoning about word meaning across languages and facilitate transfer learning for low‑resource NLP models. This survey offers a comprehensive typology of cross‑lingual word embedding models. The authors compare models’ data requirements and objective functions, discuss evaluation methods, and outline future challenges. Many models in the literature optimize the same objectives and are often equivalent, differing mainly in optimization strategies and hyper‑parameters.
Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages. In this survey, we provide a comprehensive typology of cross-lingual word embedding models. We compare their data requirements and objective functions. The recurring theme of the survey is that many of the models presented in the literature optimize for the same objectives, and that seemingly different models are often equivalent, modulo optimization strategies, hyper-parameters, and such. We also discuss the different ways cross-lingual word embeddings are evaluated, as well as future challenges and research horizons.
160
Indexing by latent semantic analysis
Scott Deerwester, Susan Dumais, George W. Furnas et al. · Journal of the American Society for Information Science · 1990 · 12.8K citations
A unified architecture for natural language processing
Ronan Collobert, Jason Weston · 2008 · 5.2K citations