2022 · 391 citations · 24 references
ComputingComputer ScienceKnowledge RepresentationMachine LearningNatural Language ProcessingLanguage ModelingLanguage ModelsNatural Language GenerationNatural Language UnderstandingAutoregressive ModelsFew-shot Language ModelingKnowledge-based InferenceLanguage Model PretrainingNeural Language Models
Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach. Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models. 2022.
24
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
Linting Xue, Noah Constant, Adam P. Roberts et al. · 2021 · 1.6K citations · Full text
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan et al. · arXiv (Cornell University) · 2020 · 1.5K citations · Full text
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase et al. · 2020 · 709 citations