Concepedia

All-reduce

All-Reduce is a collective communication operation in parallel computing that belongs to the class of reduction operations, distinguished by its requirement that the result of reducing data across all processing nodes is distributed back to all nodes, thereby ensuring uniform availability of the global reduced value.

20

Publications

1.7K

Citations

104

Authors

30

Institutions

Publications per year

2017–2026

14

Parents

Publications

20

Bandwidth optimal all-reduce algorithms for clusters of workstations

Pitch Patarasuk, Xin Yuan · Journal of Parallel and Distributed Computing · 2008 · 397 citations

, , +12

A generic communication scheduler for distributed DNN training acceleration

Yanghua Peng, Yibo Zhu, Yangrui Chen et al. · 2019 · 319 citations

, , +17

Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes

Xianyan Jia, Shutao Song, Wei He et al. · arXiv (Cornell University) · 2018 · 313 citations · Full text

, , +17

BlueConnect: Decomposing all-reduce for deep learning on heterogeneous network hierarchy

Min Chul Cho, Ulrich Finkler, Maurício Serrano et al. · IBM Journal of Research and Development · 2019 · 87 citations

, , +17

Themis

Alexander Rasmussen, Vinh The Lam, Michael Conley et al. · 2012 · 71 citations

, , +17

1–5 of 20

Authors

104

  1. YB
    Yixin Bao

    University of Hong Kong, Hong Kong SAR China

    2 publications · 348 citations

    2

    h-index

  2. YP
    Yanghua Peng

    University of Hong Kong, Hong Kong SAR China

    2 publications · 348 citations

    2

    h-index

  3. CW
    Chuan Wu

    University of Hong Kong, Hong Kong SAR China

    2 publications · 348 citations

    2

    h-index

  4. AB
    Andrew Boutros

    University of Toronto, Canada

    1 publications · 24 citations

    1

    h-index

  5. SS
    Soham Shah

    IBM Research - Austin, United States

    1 publications · 21 citations

    1

    h-index

1–5 of 104

Institutions

30

  1. University of Hong Kong

    Pok Fu Lam, Hong Kong

    2 h-index

    7 publications

    1.4K citations

  2. City University of Hong Kong

    Hong Kong, Hong Kong

    1 h-index

    2 publications

    74 citations

  3. Sun Yat-sen University

    Guangzhou, China

    1 h-index

    1 publications

    37 citations

  4. Peking University

    Beijing, China

    1 h-index

    1 publications

    37 citations

  5. IBM Research - Austin

    Austin, United States

    1 h-index

    2 publications

    42 citations

  6. University of California, Santa Barbara

    Santa Barbara, United States

    1 h-index

    1 publications

    40 citations

  7. The University of Texas at Austin

    Austin, United States

    1 h-index

    1 publications

    24 citations

  8. Knoxville College

    Knoxville, United States

    1 h-index

    1 publications

    28 citations

1–8 of 30

Venues

  1. arXiv (Cornell University)

    Repository · 3 publications · 395 citations

    3

    h-index

  2. IEEE Access

    1 publications · 34 citations

    1

    h-index

  3. Parallel Computing

    1 publications · 12 citations

    1

    h-index

  4. IEEE Computer Architecture Letters

    1 publications · 24 citations

    1

    h-index

  5. IEEE/ACM Transactions on Networking

    TON · 1 publications · 29 citations

    1

    h-index

1–5