RoFormer: Enhanced transformer with Rotary Position Embedding

HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanities Use Case

Yinhan Liu, Myle Ott, Jingfei Du et al. · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2019 · 17.3K citations · Full text

Computing, Computer Science, Computational Linguistics +13

Exploring the Limits of Transfer Learning with a Unified Text-to-Text\n Transformer

Colin Raffel, Noam Shazeer, Adam Roberts et al. · arXiv (Cornell University) · 2019 · 8.3K citations · Full text

Computing, Computer Science, Computational Linguistics +14

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Yonghui Wu, Mike Schuster, Zhifeng Chen et al. · arXiv (Cornell University) · 2016 · 5.7K citations · Full text

Computing, Computer Science, Corpus-based Translation Studies +9

The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Leo Gao, Stella Biderman, Sid Black et al. · arXiv (Cornell University) · 2020 · 487 citations · Full text

Computing, Computer Science, Computational Linguistics +10

Training Tips for the Transformer Model

Martin Popel, Ondřej Bojar · ˜The œPrague Bulletin of Mathematical Linguistics · 2018 · 280 citations · Full text

Computing, Computer Science, Corpus-based Translation Studies +14