Concepedia

Parameter Synchronization

Parameter Synchronization is a data parallelism technique that coordinates the concurrent updates of model parameters across distributed computing nodes by aggregating gradients or parameter values to maintain consistency and ensure coherent model convergence during distributed training.

55

Publications

3K

Citations

211

Authors

83

Institutions

Publications per year

2017–2026

40

Parents

Publications

55

More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server

Qirong Ho, James Cipar, Henggang Cui et al. · PubMed · 2013 · 554 citations · Full text

, , +16

Poseidon: An Efficient Communication Architecture for Distributed Deep\n Learning on GPU Clusters

Hao Zhang, Zeyu Zheng, Shizhen Xu et al. · arXiv (Cornell University) · 2017 · 198 citations · Full text

, , +17

Parallel and distributed simulation: traditional techniques and recent advances

Kalyan S. Perumalla · Winter Simulation Conference · 2006 · 111 citations

, , +17

Formation Control of Multiple Mobile Robots Incorporating an Extended State Observer and Distributed Model Predictive Approach

Andong Liu, Wen‐An Zhang, Li Yu et al. · IEEE Transactions on Systems Man and Cybernetics Systems · 2018 · 107 citations

, , +16

Round-Robin Synchronization: Mitigating Communication Bottlenecks in Parameter Servers

Chen Chen, Wei Wang, Bo Li · 2019 · 93 citations

, , +17

1–5 of 55

Authors

211

  1. QH
    Qirong Ho

    Carnegie Mellon University, United States

    7 publications · 950 citations

    7

    h-index

  2. EP
    Eric P. Xing

    Carnegie Mellon University, United States

    6 publications · 923 citations

    6

    h-index

  3. JW
    Jinliang Wei

    Carnegie Mellon University, United States

    4 publications · 315 citations

    4

    h-index

  4. PX
    Pengtao Xie

    University of California San Diego, United States

    4 publications · 276 citations

    4

    h-index

  5. SG
    Song Guo

    Hong Kong Polytechnic University, Hong Kong SAR China

    4 publications · 164 citations

    4

    h-index

1–5 of 211

Institutions

83

  1. Carnegie Mellon University

    Pittsburgh, United States

    5 h-index

    28 publications

    6.7K citations

  2. Hong Kong Polytechnic University

    Hong Kong, Hong Kong

    5 h-index

    7 publications

    284 citations

  3. University of California, Los Angeles

    Los Angeles, United States

    3 h-index

    3 publications

    99 citations

  4. Zhejiang University

    Hangzhou, China

    3 h-index

    9 publications

    176 citations

  5. Tsinghua University

    Beijing, China

    3 h-index

    16 publications

    506 citations

  6. Hong Kong University of Science and Technology

    Hong Kong, Hong Kong

    3 h-index

    9 publications

    561 citations

  7. Shanghai Jiao Tong University

    Shanghai, China

    3 h-index

    6 publications

    198 citations

  8. Hangzhou Dianzi University

    Hangzhou, China

    2 h-index

    14 publications

    175 citations

1–8 of 83

Venues

  1. arXiv (Cornell University)

    Repository · 7 publications · 392 citations

    7

    h-index

  2. Figshare

    Repository · 2 publications · 120 citations

    2

    h-index

  3. 2

    h-index

  4. IEEE Transactions on Parallel and Distributed Systems

    TPDS · 2 publications · 103 citations

    2

    h-index

  5. Mobile Information Systems

    1 publications · 13 citations

    1

    h-index

1–5