Crosslingual Generalization through Multitask Finetuning

Abstract

Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

References

Page 1

	Year	Citations
Binary codes capable of correcting deletions, insertions and reversals V.I. Levenshtein Soviet physics. Doklady EngineeringInformation RetrievalString-searching AlgorithmProgram AnalysisKnowledge Discovery	1965	10.4K
LoRA: Low-Rank Adaptation of Large Language Models J. Edward Hu, Yelong Shen, Phillip Wallis, arXiv (Cornell University) Llm Fine-tuningEngineeringMachine LearningCross-lingual RepresentationMultilingual Pretraining	2021	2.4K
Cross-lingual Language Model Pretraining Guillaume Lample, Alexis Conneau arXiv (Cornell University) Llm Fine-tuningSupervised Machine TranslationEngineeringMachine LearningCross-lingual Representation	2019	1.6K
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer Linting Xue, Noah Constant, Adam P. Roberts, EngineeringMultilingual PretrainingLanguage ProcessingNatural Language ProcessingAdam Roberts	2021	1.5K
Analysing Off-The-Shelf Options for Question Answering with Portuguese FAQs Susan Zhang, Stephen Roller, Naman Goyal, DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)	2022	906
Training Compute-Optimal Large Language Models Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, arXiv (Cornell University) Natural Language ProcessingLarge Language ModelsLarge Ai ModelLlm Fine-tuningEngineering	2022	648
Multilingual Denoising Pre-training for Neural Machine Translation Yinhan Liu, Jiatao Gu, Naman Goyal, arXiv (Cornell University) Natural Language ProcessingMultilingual Denoising Pre-trainingLlm Fine-tuningEngineeringMachine Learning	2020	607
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben Zaken, Yoav Goldberg, Shauli Ravfogel	2022	603
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng, Xiao Liu, Zhengxiao Du, arXiv (Cornell University) Llm Fine-tuningEngineeringMachine LearningMultilingualismMultilingual Pretraining	2022	293
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning Haokun Liu, Derek Tam, Jay Mohta, arXiv (Cornell University) Artificial IntelligenceFew-shot LearningStructured PredictionLlm Fine-tuningEngineering	2022	292

Page 1