Concepedia

What Language Model to Train if You Have One Million GPU Hours?

Teven Le Scao, Thomas J. Wang, Daniel Hesslow, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press,

2022 · 32 citations · 54 references

DOIFull text

Open access

Concepts

Abstract

Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy. Findings of the Association for Computational Linguistics: EMNLP 2022. 2022.

References

54

Adam: A Method for Stochastic Optimization

Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text

, , +16

MizAR 60 for Mizar 50

DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text

, , +14

Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics

DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 14.2K citations · Full text

, , +13

Exploring the Limits of Transfer Learning with a Unified Text-to-Text\n Transformer

Colin Raffel, Noam Shazeer, Adam Roberts et al. · arXiv (Cornell University) · 2019 · 8.3K citations · Full text

, , +14

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Richard Socher, Alex Perelygin, Jean Y. Wu et al. · 2013 · 6.8K citations · Full text

, , +16