Concepedia

Gradient Noise

Gradient Noise is a stochastic perturbation inherent in the estimated gradient during gradient-based optimization, arising from the use of noisy or approximate gradient calculations, which distinguishes it from deterministic gradient errors by its random nature and typically zero-mean distribution.

149

Publications

7.9K

Citations

460

Authors

185

Institutions

Publications per year

2017–2026

94

Parents

Publications

149

Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks

Jianshu Chen, Ali H. Sayed · IEEE Transactions on Signal Processing · 2012 · 693 citations · Full text

, , +16

On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima

Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal et al. · arXiv (Cornell University) · 2016 · 579 citations · Full text

, , +12

A Stochastic Quasi-Newton Method for Large-Scale Optimization

Richard H. Byrd, Samantha Hansen, Jorge Nocedal et al. · SIAM Journal on Optimization · 2016 · 452 citations

, , +16

Adding Gradient Noise Improves Learning for Very Deep Networks

Arvind Neelakantan, Luke Vilnis, Quoc V. Le et al. · arXiv (Cornell University) · 2015 · 267 citations · Full text

, , +15

Federated Variance-Reduced Stochastic Gradient Descent With Robustness to Byzantine Attacks

Zhaoxian Wu, Qing Ling, Tianyi Chen et al. · IEEE Transactions on Signal Processing · 2020 · 215 citations

, , +16

1–5 of 149

Authors

460

  1. AH
    Ali H. Sayed

    University of California, Los Angeles, United States

    7 publications · 818 citations

    7

    h-index

  2. MG
    Mert Gürbüzbalaban

    Rutgers Sexual and Reproductive Health and Rights, Netherlands

    6 publications · 188 citations

    6

    h-index

  3. SV
    Stefan Vlaski

    École Polytechnique Fédérale de Lausanne, Switzerland

    6 publications · 125 citations

    6

    h-index

  4. JN
    Jorge Nocedal

    Northwestern University, United States

    5 publications · 1.2K citations

    5

    h-index

  5. UŞ
    Umut Şimşekli

    Laboratoire Traitement et Communication de l’Information, France

    5 publications · 177 citations

    5

    h-index

1–5 of 460

Institutions

185

  1. École Polytechnique Fédérale de Lausanne

    Lausanne, Switzerland

    8 h-index

    14 publications

    352 citations

  2. University of California, Los Angeles

    Los Angeles, United States

    7 h-index

    11 publications

    1.6K citations

  3. Johns Hopkins University

    Baltimore, United States

    4 h-index

    7 publications

    202 citations

  4. Massachusetts Institute of Technology

    Cambridge, United States

    4 h-index

    7 publications

    205 citations

  5. University of California, Berkeley

    Berkeley, United States

    4 h-index

    9 publications

    510 citations

  6. California Institute of Technology

    Pasadena, United States

    4 h-index

    5 publications

    448 citations

  7. Northwestern University

    Evanston, United States

    4 h-index

    7 publications

    1.5K citations

  8. Princeton University

    Princeton, United States

    4 h-index

    6 publications

    191 citations

1–8 of 185

Venues

  1. arXiv (Cornell University)

    Repository · 41 publications · 2.4K citations

    24

    h-index

  2. Magnetic Resonance in Medicine

    MRM · 6 publications · 481 citations

    6

    h-index

  3. IEEE Transactions on Signal Processing

    6 publications · 1K citations

    6

    h-index

  4. SIAM Journal on Optimization

    5 publications · 542 citations

    5

    h-index

  5. 4

    h-index

1–5