Concepedia

Positional Encoding

Positional Encoding is a deterministic or learned vector representation added to token embeddings in sequence models to provide information about the relative or absolute positions of tokens within a sequence, distinguishing it from other embedding components by explicitly encoding sequential order in contexts where such information is otherwise absent.

317

Publications

35.6K

Citations

1.1K

Authors

389

Institutions

Publications per year

2017–2026

206

Parents

Publications

317

SegFormer: Simple and Efficient Design for Semantic Segmentation with\n Transformers

Enze Xie, Wenhai Wang, Zhiding Yu et al. · arXiv (Cornell University) · 2021 · 3.3K citations · Full text

, , +14

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

Zihang Dai, Zhilin Yang, Yiming Yang et al. · 2019 · 3.1K citations · Full text

, , +16

Positional information and the spatial pattern of cellular differentiation

Lewis Wolpert · Journal of Theoretical Biology · 1969 · 3.1K citations

, , +16

Are Transformers Effective for Time Series Forecasting?

Ailing Zeng, Muxi Chen, Lei Zhang et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 2023 · 2.6K citations · Full text

, , +15

RoFormer: Enhanced transformer with Rotary Position Embedding

Jianlin Su, Murtadha Ahmed, Yu Lu et al. · Neurocomputing · 2023 · 1.4K citations

, , +16

1–5 of 317

Authors

1.1K

  1. LW
    Lewis Wolpert

    University College London, United Kingdom

    7 publications · 3.7K citations

    7

    h-index

  2. SJ
    Sen Jia

    University of Bristol, United Kingdom

    4 publications · 191 citations

    4

    h-index

  3. ND
    Neil D. B. Bruce

    University of Manitoba, Canada

    4 publications · 191 citations

    4

    h-index

  4. DC
    Dongdong Chen

    University of Science and Technology of China, China

    4 publications · 1.7K citations

    4

    h-index

  5. PW
    Peter W. Reddien

    Howard Hughes Medical Institute, United States

    3 publications · 193 citations

    3

    h-index

1–5 of 1.1K

Institutions

389

  1. University College London

    London, United Kingdom

    8 h-index

    10 publications

    1.2K citations

  2. Howard Hughes Medical Institute

    Chevy Chase, United States

    8 h-index

    16 publications

    1.5K citations

  3. Tsinghua University

    Beijing, China

    5 h-index

    12 publications

    356 citations

  4. National University of Singapore

    Singapore, Singapore

    5 h-index

    11 publications

    807 citations

  5. Centre National de la Recherche Scientifique

    Paris, France

    5 h-index

    17 publications

    3.6K citations

  6. Peking University

    Beijing, China

    5 h-index

    7 publications

    433 citations

  7. Google (United States)

    Mountain View, United States

    5 h-index

    25 publications

    6.3K citations

  8. Massachusetts Institute of Technology

    Cambridge, United States

    5 h-index

    12 publications

    651 citations

1–8 of 389

Venues

  1. arXiv (Cornell University)

    Repository · 49 publications · 6.9K citations

    28

    h-index

  2. Developmental Biology

    10 publications · 483 citations

    10

    h-index

  3. Proceedings of the AAAI Conference on Artificial Intelligence

    Conference · 9 publications · 3.2K citations

    9

    h-index

  4. 9

    h-index

  5. Development

    6 publications · 208 citations

    6

    h-index

1–5