Concepedia

Transformer Models

Transformer Models is a neural network architecture distinguished by its use of self-attention mechanisms to process sequential data in parallel, enabling highly effective modeling of long-range dependencies in time series prediction and other sequence-based tasks.

5.6K

Publications

493.8K

Citations

17.1K

Authors

2.8K

Institutions

Publications per year

2017–2026

4.9K

Variants

Transformers

Parents

Publications

5.6K

MizAR 60 for Mizar 50

DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text

, , +14

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Ze Liu, Yutong Lin, Yue Cao et al. · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · 2021 · 30.1K citations

, , +16

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov et al. · arXiv (Cornell University) · 2020 · 21.6K citations · Full text

, , +14

Attention Is All You Need

Ashish Vaswani, Noam Shazeer, Niki Parmar et al. · 2025 · 6.6K citations · Full text

, , +14

Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

Haoyi Zhou, Shanghang Zhang, Jieqi Peng et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 2021 · 6.1K citations · Full text

, , +14

1–5 of 5.6K

Authors

17.1K

  1. SW
    Shinji Watanabe

    Johns Hopkins University, United States

    20 publications · 2K citations

    17

    h-index

  2. FW
    Furu Wei

    Microsoft Research Asia (China), China

    21 publications · 3.6K citations

    16

    h-index

  3. TQ
    Tao Qin

    Microsoft Research Asia (China), China

    16 publications · 1.8K citations

    15

    h-index

  4. YT
    Yi Tay

    Nanyang Technological University, Singapore

    14 publications · 1.1K citations

    14

    h-index

  5. SL
    Shujie Liu

    Microsoft Research Asia (China), China

    14 publications · 3.1K citations

    14

    h-index

1–5 of 17.1K

Institutions

2.8K

  1. Google (United States)

    Mountain View, United States

    53 h-index

    450 publications

    324.5K citations

  2. Chinese Academy of Sciences

    Beijing, China

    45 h-index

    314 publications

    20.9K citations

  3. Tsinghua University

    Beijing, China

    44 h-index

    255 publications

    58.5K citations

  4. University of Chinese Academy of Sciences

    Beijing, China

    37 h-index

    194 publications

    14.9K citations

  5. Meta (Israel)

    Tel Aviv, Israel

    36 h-index

    188 publications

    22.3K citations

  6. Shanghai Jiao Tong University

    Shanghai, China

    36 h-index

    187 publications

    18.6K citations

  7. Peking University

    Beijing, China

    35 h-index

    148 publications

    11.2K citations

  8. Zhejiang University

    Hangzhou, China

    34 h-index

    141 publications

    11.6K citations

1–8 of 2.8K

Venues

  1. arXiv (Cornell University)

    Repository · 819 publications · 91K citations

    114

    h-index

  2. IEEE Transactions on Power Delivery

    147 publications · 13.1K citations

    58

    h-index

  3. 54

    h-index

  4. Proceedings of the AAAI Conference on Artificial Intelligence

    Conference · 95 publications · 18.3K citations

    46

    h-index

  5. IEEE Access

    98 publications · 5K citations

    41

    h-index

1–5