Concepedia

All-reduce Operations

All-Reduce Operations is a collective communication primitive in distributed memory programming that belongs to the class of reduction operations and is distinguished by its requirement to compute a global reduction (such as sum, max, or product) of data across all processing elements and then distribute the resulting value to every participant in the system.

14

Publications

572

Citations

62

Authors

22

Institutions

Publications per year

2017–2026

10

Parents

Publications

14

BlueConnect: Decomposing all-reduce for deep learning on heterogeneous network hierarchy

Min Chul Cho, Ulrich Finkler, Maurício Serrano et al. · IBM Journal of Research and Development · 2019 · 87 citations

, , +17

Randomized Distributed Mean Estimation: Accuracy vs. Communication

Jakub Konečný, Peter Richtárik · Frontiers in Applied Mathematics and Statistics · 2018 · 74 citations · Full text

, , +17

An In-Network Architecture for Accelerating Shared-Memory Multiprocessor Collectives

Benjamin Klenk, Nan Jiang, Greg Thorson et al. · 2020 · 64 citations

, , +17

Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce

Xupeng Miao, Xiaonan Nie, Yingxia Shao et al. · 2021 · 61 citations

, , +17

How to scale distributed deep learning?

Peter Jin, Qiaochu Yuan, Forrest Iandola et al. · arXiv (Cornell University) · 2016 · 53 citations · Full text

, , +17

1–5 of 14

Authors

62

  1. JK
    Jakub Konečný

    University of Edinburgh, United Kingdom

    1 publications · 74 citations

    1

    h-index

  2. GT
    Greg Thorson

    Pacific Northwest National Laboratory, United States

    1 publications · 64 citations

    1

    h-index

  3. WW
    Wenfei Wu

    Tsinghua University, China

    1 publications · 37 citations

    1

    h-index

  4. AM
    Abdullah Muzahid

    University of Illinois Urbana-Champaign, United States

    1 publications · 40 citations

    1

    h-index

  5. SS
    Soham Shah

    IBM Research - Austin, United States

    1 publications · 21 citations

    1

    h-index

1–5 of 62

Institutions

22

  1. Peking University

    Beijing, China

    2 h-index

    6 publications

    342 citations

  2. King Abdullah University of Science and Technology

    Jeddah, Saudi Arabia

    2 h-index

    2 publications

    111 citations

  3. Tencent (China)

    Shenzhen, China

    1 h-index

    1 publications

    61 citations

  4. University of California, Santa Barbara

    Santa Barbara, United States

    1 h-index

    1 publications

    40 citations

  5. University of Vienna

    Vienna, Austria

    1 h-index

    3 publications

    36 citations

  6. Yonsei University

    Seoul, South Korea

    1 h-index

    1 publications

    21 citations

  7. Texas A&M University

    College Station, United States

    1 h-index

    5 publications

    200 citations

  8. City University of Hong Kong

    Hong Kong, Hong Kong

    1 h-index

    2 publications

    74 citations

1–8 of 22

Venues

  1. IEEE Access

    1 publications · 34 citations

    1

    h-index

  2. 1

    h-index

  3. ACM Transactions on Parallel Computing

    TOPC · 1 publications · 14 citations

    1

    h-index

  4. IEEE Computer Architecture Letters

    1 publications · 24 citations

    1

    h-index

  5. Parallel Computing

    1 publications · 12 citations

    1

    h-index

1–5