Concepedia

Synchronous Data Parallelism

Synchronous Data Parallelism is a data parallelism strategy in which all processing units simultaneously execute the same operation across different data partitions, coordinating their progress through global synchronization barriers at each computational step.

33

Publications

1.8K

Citations

124

Authors

50

Institutions

Publications per year

2017–2026

16

Parents

Publications

33

ImageNet Training in Minutes

Yang You, Zhao Zhang, Cho‐Jui Hsieh et al. · 2018 · 408 citations

, , +15

DAPPLE

Shiqing Fan, Yi Rong, Meng Chen et al. · 2021 · 197 citations

, , +17

A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networks

Shaohuai Shi, Qiang Wang, Kaiyong Zhao et al. · 2019 · 163 citations

, , +16

A unified approach to models of synchronous parallel machines

Leslie M. Goldschlager · 1978 · 127 citations · Full text

, , +10

CosmoFlow: Using Deep Learning to Learn the Universe at Scale

Amrita Mathuriya, Deborah Bard, P. J. Mendygral et al. · 2018 · 122 citations · Full text

, , +17

1–5 of 33

Authors

124

  1. YY
    Yang You

    National University of Singapore, Singapore

    3 publications · 491 citations

    3

    h-index

  2. CH
    Cho‐Jui Hsieh

    University of California, Los Angeles, United States

    3 publications · 491 citations

    3

    h-index

  3. JD
    James Demmel

    University of California, Berkeley, United States

    3 publications · 491 citations

    3

    h-index

  4. XC
    Xiaowen Chu

    Hong Kong Baptist University, Hong Kong SAR China

    2 publications · 197 citations

    2

    h-index

  5. JA
    James Arnemann

    Flatiron Health (United States), United States

    2 publications · 146 citations

    2

    h-index

1–5 of 124

Institutions

50

  1. University of California, Berkeley

    Berkeley, United States

    5 h-index

    9 publications

    1.5K citations

  2. University of California, Davis

    Davis, United States

    2 h-index

    2 publications

    453 citations

  3. Alibaba Group (United States)

    San Mateo, United States

    2 h-index

    24 publications

    2.7K citations

  4. University of Hong Kong

    Pok Fu Lam, Hong Kong

    2 h-index

    2 publications

    226 citations

  5. University of California San Diego

    San Diego, United States

    2 h-index

    2 publications

    67 citations

  6. University of Washington

    Seattle, United States

    2 h-index

    2 publications

    67 citations

  7. Berkeley College

    New York, United States

    2 h-index

    6 publications

    1.4K citations

  8. Hong Kong Baptist University

    Hong Kong, Hong Kong

    2 h-index

    7 publications

    1K citations

1–8 of 50

Venues

  1. arXiv (Cornell University)

    Repository · 5 publications · 155 citations

    5

    h-index

  2. Cybernetics and Systems Analysis

    1 publications · 35 citations

    1

    h-index

  3. 1

    h-index

  4. ACM Transactions on Architecture and Code Optimization

    TACO · 1 publications · 10 citations

    1

    h-index

  5. 1

    h-index

1–5