Gradient Descent

Gradient Descent is an iterative optimization algorithm that belongs to the class of first-order methods, distinguished by its use of the negative gradient of the objective function to update parameters in the direction of steepest descent for minimizing loss in machine learning models.

7.8K

Publications

821.9K

Citations

17.1K

Authors

3.1K

Institutions

Publications per year

2017–2026

4.2K

Variants

Method Of Steepest Descent

Parents

Continuous OptimizationIterative SolversLarge-scale OptimizationOptimization For Machine LearningUnconstrained OptimizationVariational Optimization

Publications

7.8K

Adam: A Method for Stochastic Optimization

Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text

Computing, Computer Science, Algorithms +16

Greedy function approximation: A gradient boosting machine.

Jerome H. Friedman · The Annals of Statistics · 2001 · 29K citations · Full text

Computing, Computer Science, Mathematics +17

Reducing the Dimensionality of Data with Neural Networks

Geoffrey E. Hinton, Ruslan Salakhutdinov · Science · 2006 · 21K citations

Computing, Computer Science, Artificial Intelligence +14

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

Stephen Boyd · Foundations and Trends® in Machine Learning · 2011 · 16K citations

Computing, Computer Science, Machine Learning +13

Understanding the difficulty of training deep feedforward neural networks

Xavier Glorot, Yoshua Bengio · 2010 · 12.7K citations

Computing, Computer Science, Artificial Intelligence +13

1–5 of 7.8K

Authors

17.1K

YB
Yoshua Bengio
Université de Montréal, Canada
35 publications · 26.1K citations
27
h-index
MI
Michael I. Jordan
University of California, Berkeley, United States
36 publications · 3K citations
27
h-index
JD
Jason D. Lee
University of Southern California, United States
33 publications · 2.6K citations
24
h-index
ZA
Zeyuan Allen-Zhu
Princeton University, United States
26 publications · 2.5K citations
22
h-index
QG
Quanquan Gu
University of California, Los Angeles, United States
32 publications · 2.4K citations
22
h-index

1–5 of 17.1K

Institutions

3.1K

1–8 of 3.1K

Venues

arXiv (Cornell University)
Repository · 1.4K publications · 136.5K citations
138
h-index
IEEE Transactions on Neural Networks
91 publications · 18.3K citations
55
h-index
SIAM Journal on Optimization
95 publications · 13.8K citations
52
h-index
IEEE Transactions on Neural Networks and Learning Systems
97 publications · 6.5K citations
45
h-index
Neural Information Processing Systems
Conference · 98 publications · 9.9K citations
44
h-index

1–5