Transformers: State-of-the-Art Natural Language Processing

Thomas Wolf, Lysandre Debut, Victor Sanh, +19

EngineeringLanguage LearningLanguage ProcessingText MiningNatural Language Processing +19

Transformers: State-of-the-Art Natural Language Processing

Thomas Wolf, Lysandre Debut, Victor Sanh, +13

Zenodo (CERN European Organization for Nuclear Research)

Multitask Prompted Training Enables Zero-Shot Task Generalization

Victor Sanh, Albert Webson, Colin Raffel, +24

arXiv (Cornell University)

Artificial IntelligenceFew-shot LearningLlm Fine-tuningEngineeringMachine Learning +18

Datasets: A Community Library for Natural Language Processing

Quentin Lhoest, A. Villanova del Moral, Yacine Jernite, +29

EngineeringCommunity LibrarySystem DemonstrationsQuentin LhoestLanguage Processing +16

Crosslingual Generalization through Multitask Finetuning

Niklas Muennighoff, Thomas J. Wang, Lintang Sutawika, +16

EngineeringCross-lingual RepresentationCrosslingual GeneralizationLanguage ProcessingText Mining +17

Mistral 7B

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, +15

arXiv (Cornell University)

Artificial IntelligenceLlm Fine-tuningEngineeringMachine LearningMistral 7B +15

Mixtral of Experts

Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, +23

arXiv (Cornell University)

Artificial IntelligenceLlm Fine-tuningEngineeringMachine LearningSparse Mixture +15

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

Hugo Laurençon, Lucile Saulnier, Thomas J. Wang, +51

arXiv (Cornell University)

EngineeringMultilingual PretrainingLarge Language ModelLarge-scale DatasetsCorpus Linguistics +15

Scaling Data-Constrained Language Models

Niklas Muennighoff, Alexander M. Rush, Boaz Barak, +6

arXiv (Cornell University)

Llm Fine-tuningEngineeringMachine LearningMultilingual PretrainingLarge Language Model +19

What Language Model to Train if You Have One Million GPU Hours?

Teven Le Scao, Thomas J. Wang, Daniel Hesslow, +15

Artificial IntelligenceEngineeringMachine LearningComputer ArchitectureCommunication +27