Concepedia

Next-token Prediction

Next-Token Prediction is a computational task in language modeling where the objective is to predict the most probable subsequent token in a sequence given its preceding context, serving as the foundational training paradigm for autoregressive language models.

68

Publications

7.3K

Citations

1.1K

Authors

83

Institutions

Publications per year

2017–2026

63

Variants

Next Token Prediction

Parents

Publications

68

GPT-4 Technical Report

arXiv (Cornell University) · 2023 · 2.4K citations · Full text

, , +16

Federated Learning for Mobile Keyboard Prediction

Andrew Hard, K. Praveen Kumar Rao, Rajiv Mathews et al. · arXiv (Cornell University) · 2018 · 1.1K citations · Full text

, , +17

Toolformer: Language Models Can Teach Themselves to Use Tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dessì et al. · arXiv (Cornell University) · 2023 · 393 citations · Full text

, , +13

ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training

Weizhen Qi, Yu Yan, Yeyun Gong et al. · 2020 · 340 citations · Full text

, , +16

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Gemini Robotics Team, Petko Georgiev, Ving Ian Lei et al. · arXiv (Cornell University) · 2024 · 285 citations · Full text

, , +16

1–5 of 68

Authors

1.1K

  1. PG
    Petko Georgiev

    University of Cambridge, United Kingdom

    2 publications · 419 citations

    2

    h-index

  2. EF
    Evelina Fedorenko

    Massachusetts Institute of Technology, United States

    2 publications · 65 citations

    2

    h-index

  3. MZ
    Ming Zhang

    Chinese Academy of Sciences, China

    2 publications · 419 citations

    2

    h-index

  4. YW
    Yonghui Wu

    Google (United States), United States

    2 publications · 377 citations

    2

    h-index

  5. DR
    Dominika Rogozińska

    Heliodor Swiecicki Clinical Hospital, Poland

    2 publications · 419 citations

    2

    h-index

1–5 of 1.1K

Institutions

83

  1. Princeton University

    Princeton, United States

    3 h-index

    11 publications

    482 citations

  2. Stanford University

    Stanford, United States

    3 h-index

    6 publications

    145 citations

  3. Columbia University

    New York, United States

    2 h-index

    3 publications

    220 citations

  4. New York University

    New York, United States

    2 h-index

    2 publications

    65 citations

  5. Peking University

    Beijing, China

    2 h-index

    5 publications

    681 citations

  6. International Institute of Information Technology, Hyderabad

    Hyderabad, India

    2 h-index

    6 publications

    87 citations

  7. University College London

    London, United Kingdom

    2 h-index

    8 publications

    205 citations

  8. Zhejiang University

    Hangzhou, China

    2 h-index

    13 publications

    453 citations

1–8 of 83

Venues

  1. arXiv (Cornell University)

    Repository · 29 publications · 5.4K citations

    20

    h-index

  2. ChemRxiv

    Repository · 2 publications · 29 citations

    2

    h-index

  3. IEEE Intelligent Systems

    1 publications · 114 citations

    1

    h-index

  4. Nature Machine Intelligence

    1 publications · 24 citations

    1

    h-index

  5. 1

    h-index

1–5