Concepedia

All-reduce Algorithms

All-Reduce Algorithms is a collective communication paradigm in cluster interconnects and cluster networking that belongs to the class of distributed synchronization methods, distinguished by its ability to perform a global reduction operation across all processing nodes and subsequently broadcast the result to every node in the system.

39

Publications

3.1K

Citations

142

Authors

51

Institutions

Publications per year

2017–2026

20

Parents

Publications

39

Optimization of Collective Communication Operations in MPICH

Rajeev Thakur, Rolf Rabenseifner, William Gropp · The International Journal of High Performance Computing Applications · 2005 · 909 citations

, , +17

Bandwidth optimal all-reduce algorithms for clusters of workstations

Pitch Patarasuk, Xin Yuan · Journal of Parallel and Distributed Computing · 2008 · 397 citations

, , +12

Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes

Xianyan Jia, Shutao Song, Wei He et al. · arXiv (Cornell University) · 2018 · 313 citations · Full text

, , +17

2020 IEEE 26th International Conference on Parallel and Distributed Systems

2020 · 112 citations · Full text

, , +16

BlueConnect: Decomposing all-reduce for deep learning on heterogeneous network hierarchy

Min Chul Cho, Ulrich Finkler, Maurício Serrano et al. · IBM Journal of Research and Development · 2019 · 87 citations

, , +17

1–5 of 39

Authors

142

  1. WG
    William Gropp

    Argonne National Laboratory, United States

    4 publications · 1K citations

    4

    h-index

  2. DK
    Dhabaleswar K. Panda

    The Ohio State University, United States

    3 publications · 86 citations

    3

    h-index

  3. YT
    Yoshiki Tanaka

    Nara Institute of Science and Technology, Japan

    2 publications · 130 citations

    2

    h-index

  4. HS
    Hisahiro Suganuma

    2 publications · 130 citations

    2

    h-index

  5. PS
    Paul Sack

    University of Illinois Urbana-Champaign, United States

    2 publications · 45 citations

    2

    h-index

1–5 of 142

Institutions

51

  1. Argonne National Laboratory

    Lemont, United States

    3 h-index

    5 publications

    1.9K citations

  2. University of Illinois Urbana-Champaign

    Urbana, United States

    2 h-index

    4 publications

    90 citations

  3. King Abdullah University of Science and Technology

    Jeddah, Saudi Arabia

    2 h-index

    2 publications

    111 citations

  4. Peking University

    Beijing, China

    2 h-index

    6 publications

    342 citations

  5. The Ohio State University

    Columbus, United States

    2 h-index

    5 publications

    164 citations

  6. University of Hong Kong

    Pok Fu Lam, Hong Kong

    2 h-index

    5 publications

    275 citations

  7. The University of Texas at Austin

    Austin, United States

    2 h-index

    3 publications

    146 citations

  8. Florida State University

    Tallahassee, United States

    2 h-index

    4 publications

    870 citations

1–8 of 51

Venues

  1. arXiv (Cornell University)

    Repository · 4 publications · 496 citations

    4

    h-index

  2. IEEE Computer Architecture Letters

    1 publications · 24 citations

    1

    h-index

  3. IBM Journal of Research and Development

    1 publications · 87 citations

    1

    h-index

  4. Journal of Parallel and Distributed Computing

    1 publications · 397 citations

    1

    h-index

  5. 1

    h-index

1–5