Concepedia

Stochastic Gradient Descent

Stochastic Gradient Descent is an iterative optimization algorithm in the class of first-order gradient-based methods for unconstrained continuous optimization that approximates the true gradient of the objective function using a single randomly selected data point or mini-batch at each iteration, thereby reducing computational cost and enabling convergence in large-scale or online learning settings.

4.9K

Publications

518.8K

Citations

10.4K

Authors

2.2K

Institutions

Publications per year

2017–2026

3K

Variants

Sgd

Parents

Publications

4.9K

Adam: A Method for Stochastic Optimization

Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text

, , +16

Auto-Encoding Variational Bayes

Diederik P. Kingma, Max Welling · UvA-DARE (University of Amsterdam) · 2013 · 15.6K citations · Full text

, , +16

Diagnosing Non-Intermittent Anomalies in Reinforcement Learning Policy Executions (Short Paper)

arXiv (Cornell University) · 2017 · 11.3K citations · Full text

, , +16

Decoupled Weight Decay Regularization

Ilya Loshchilov, Frank Hutter · arXiv (Cornell University) · 2017 · 9.1K citations · Full text

, , +16

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

John C. Duchi, Elad Hazan, Yoram Singer · 2010 · 8.6K citations

, , +15

1–5 of 4.9K

Authors

10.4K

  1. FD
    Feng Ding

    Jiangnan University, China

    42 publications · 2.7K citations

    28

    h-index

  2. PR
    Peter Richtárik

    University of Edinburgh, United Kingdom

    60 publications · 3K citations

    25

    h-index

  3. MI
    Michael I. Jordan

    University of California, Berkeley, United States

    34 publications · 2K citations

    24

    h-index

  4. YB
    Yoshua Bengio

    Université de Montréal, Canada

    23 publications · 9.4K citations

    21

    h-index

  5. TY
    Tianbao Yang

    University of Iowa, United States

    37 publications · 1.4K citations

    21

    h-index

1–5 of 10.4K

Institutions

2.2K

  1. Google (United States)

    Mountain View, United States

    52 h-index

    272 publications

    108.1K citations

  2. University of California, Berkeley

    Berkeley, United States

    47 h-index

    231 publications

    33.6K citations

  3. Stanford University

    Stanford, United States

    46 h-index

    195 publications

    18.9K citations

  4. Carnegie Mellon University

    Pittsburgh, United States

    43 h-index

    197 publications

    17.1K citations

  5. Princeton University

    Princeton, United States

    42 h-index

    114 publications

    28.3K citations

  6. Massachusetts Institute of Technology

    Cambridge, United States

    41 h-index

    144 publications

    20.5K citations

  7. Microsoft (United States)

    Redmond, United States

    34 h-index

    109 publications

    11.6K citations

  8. Jiangnan University

    Wuxi, China

    33 h-index

    128 publications

    8.5K citations

1–8 of 2.2K

Venues

  1. arXiv (Cornell University)

    Repository · 1.3K publications · 130.9K citations

    137

    h-index

  2. IEEE Transactions on Signal Processing

    121 publications · 11.2K citations

    52

    h-index

  3. Neural Information Processing Systems

    Conference · 82 publications · 9.3K citations

    42

    h-index

  4. SIAM Journal on Optimization

    59 publications · 7.8K citations

    38

    h-index

  5. IEEE Transactions on Automatic Control

    46 publications · 5.7K citations

    31

    h-index

1–5