2020 · 27 citations · 44 references
ComputingComputer ScienceComputational LinguisticsKnowledge RepresentationNatural Language ProcessingGraph-based LearningLanguage ModelingLanguage TransferMeta-learningMultilingual Natural Language ProcessingTransfer LearningBenchmark DatasetsCross-lingual Language ModelingCross-lingual Representation LearningCross-lingual TransferCross-lingual Transfer LearningLanguage Model PretrainingTask AdaptationTransferable Representation Learning
The recent emergence of multilingual pretraining language model (mPLM) has enabled breakthroughs on various downstream cross-lingual transfer (CLT) tasks. However, mPLM-based methods usually involve two problems: (1) simply fine-tuning may not adapt general-purpose multilingual representations to be task-aware on low-resource languages; (2) ignore how cross-lingual adaptation happens for downstream tasks. To address the issues, we propose a meta graph learning (MGL) method. Unlike prior works that transfer from scratch, MGL can learn to cross-lingual transfer by extracting meta-knowledge from historical CLT experiences (tasks), making mPLM insensitive to low-resource languages. Besides, for each CLT task, MGL formulates its transfer process as information propagation over a dynamic graph, where the geometric structure can automatically capture intrinsic language relationships to guide cross-lingual transfer explicitly. Empirically, extensive experiments on both public and real-world datasets demonstrate the effectiveness of the MGL method. © 2020 Association for Computational Linguistics
44
Prototypical Networks for Few-shot Learning
Jake Snell, Kevin Swersky, Richard S. Zemel · arXiv (Cornell University) · 2017 · 5.2K citations · Full text
Learning with Local and Global Consistency
Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal et al. · MPG.PuRe (Max Planck Society) · 2003 · 3.8K citations · Full text