Concepedia

Asynchronous Data Parallelism

Asynchronous Data Parallelism is a data parallelism strategy in which model replicas independently perform computations on local data shards and update shared parameters without requiring synchronization across all replicas at each iteration, thereby allowing for varying update frequencies and reducing idle time in distributed training environments.

16

Publications

1.1K

Citations

76

Authors

27

Institutions

Publications per year

2017–2026

7

Parents

Publications

16

NVIDIA A100 Tensor Core GPU: Performance and Innovation

Jack Choquette, Wishwesh Gandhi, Olivier Giroux et al. · IEEE Micro · 2021 · 380 citations

, , +17

Combining in-situ and in-transit processing to enable extreme-scale scientific analysis

Janine Camille Bennett, Hasan Abbasi, Peer‐Timo Bremer et al. · 2012 International Conference for High Performance Computing, Networking, Storage and Analysis · 2012 · 137 citations · Full text

, , +16

CellSort: high performance sorting on the cell processor

Buğra Gedik, Rajesh Bordawekar, Philip S. Yu · 2007 · 117 citations

, , +17

Salient features of an executable specification language and its environment

Pamela Zave, W.M. Schell · IEEE Transactions on Software Engineering · 1986 · 110 citations

, , +16

pySOT and POAP: An event-driven asynchronous framework for surrogate optimization

David Eriksson, David Bindel, Christine A. Shoemaker · arXiv (Cornell University) · 2019 · 68 citations · Full text

, , +14

1–5 of 16

Authors

76

  1. BH
    Bingsheng He

    National University of Singapore, Singapore

    1 publications · 16 citations

    1

    h-index

  2. MP
    Manish Parashar

    Rutgers, The State University of New Jersey, United States

    1 publications · 137 citations

    1

    h-index

  3. JL
    Jiali Li

    Argonne National Laboratory, United States

    1 publications · 57 citations

    1

    h-index

  4. JM
    José María González-Linares

    Universidad de Málaga, Spain

    1 publications · 41 citations

    1

    h-index

  5. TA
    Tarek Abdelzaher

    University of Illinois Urbana-Champaign, United States

    1 publications · 11 citations

    1

    h-index

1–5 of 76

Institutions

27

  1. Argonne National Laboratory

    Lemont, United States

    2 h-index

    5 publications

    268 citations

  2. University of California, Berkeley

    Berkeley, United States

    1 h-index

    4 publications

    56 citations

  3. Rutgers, The State University of New Jersey

    New Brunswick, United States

    1 h-index

    3 publications

    411 citations

  4. Microsoft (United States)

    Redmond, United States

    1 h-index

    3 publications

    33 citations

  5. Universidad de Málaga

    Málaga, Spain

    1 h-index

    2 publications

    82 citations

  6. NEC (Japan)

    Tokyo, Japan

    1 h-index

    6 publications

    180 citations

  7. AT&T (United States)

    Bedminster, United States

    1 h-index

    2 publications

    220 citations

  8. Barcelona Supercomputing Center

    Barcelona, Spain

    1 h-index

    9 publications

    189 citations

1–8 of 27

Venues

  1. IEEE Transactions on Nuclear Science

    1 publications · 14 citations

    1

    h-index

  2. Journal of Parallel and Distributed Computing

    1 publications · 41 citations

    1

    h-index

  3. Journal of Synchrotron Radiation

    1 publications · 40 citations

    1

    h-index

  4. IEEE Transactions on Software Engineering

    TSE · 1 publications · 110 citations

    1

    h-index

  5. Real-Time Systems

    1 publications · 11 citations

    1

    h-index

1–5