Concepedia

Batch Gradient Descent

Batch Gradient Descent is a gradient-based optimization algorithm that, as a member of the family of first-order iterative optimization methods, computes the gradient of the objective function with respect to the model parameters using the entire training dataset at each iteration, thereby distinguishing itself from stochastic and mini-batch variants by its full-batch gradient computation.

165

Publications

10.7K

Citations

445

Authors

198

Institutions

Publications per year

2017–2026

105

Variants

Vanilla Gradient Descent

Parents

Publications

165

Efficient projections onto the<i>l</i><sub>1</sub>-ball for learning in high dimensions

John C. Duchi, Shai Shalev‐Shwartz, Yoram Singer et al. · 2008 · 1.2K citations

, , +12

Efficient Online and Batch Learning Using Forward Backward Splitting

John C. Duchi, Yoram Singer · 2009 · 507 citations

, , +13

The general inefficiency of batch training for gradient descent learning

D.R. Wilson, Tony Martinez · Neural Networks · 2003 · 419 citations

, , +13

A Proof of Local Convergence for the Adam Optimizer

Sebastian Bock, Martin Weis · 2019 · 325 citations

, , +14

On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning

Richard H. Byrd, Gillian M. Chin, Will Neveitt et al. · SIAM Journal on Optimization · 2011 · 253 citations

, , +17

1–5 of 165

Authors

445

  1. WW
    Wei Wu

    Dalian University of Technology, China

    6 publications · 220 citations

    6

    h-index

  2. LB
    Léon Bottou

    AT&T (United States), United States

    5 publications · 317 citations

    5

    h-index

  3. ZL
    Zhiyuan Li

    Princeton University, United States

    4 publications · 104 citations

    4

    h-index

  4. KL
    Kaifeng Lyu

    Princeton University, United States

    4 publications · 104 citations

    4

    h-index

  5. SA
    Sanjeev Arora

    Princeton University, United States

    4 publications · 104 citations

    4

    h-index

1–5 of 445

Institutions

198

  1. Dalian University of Technology

    Dalian, China

    9 h-index

    24 publications

    890 citations

  2. Carnegie Mellon University

    Pittsburgh, United States

    8 h-index

    18 publications

    1.3K citations

  3. Google (United States)

    Mountain View, United States

    5 h-index

    8 publications

    4.4K citations

  4. Sun Yat-sen University

    Guangzhou, China

    5 h-index

    6 publications

    100 citations

  5. Tsinghua University

    Beijing, China

    4 h-index

    4 publications

    307 citations

  6. Stanford University

    Stanford, United States

    4 h-index

    5 publications

    231 citations

  7. Princeton University

    Princeton, United States

    4 h-index

    6 publications

    129 citations

  8. Dalian Maritime University

    Dalian, China

    4 h-index

    7 publications

    160 citations

1–8 of 198

Venues

  1. arXiv (Cornell University)

    Repository · 51 publications · 3K citations

    28

    h-index

  2. Neurocomputing

    6 publications · 184 citations

    5

    h-index

  3. Neural Networks

    4 publications · 686 citations

    4

    h-index

  4. IEEE Access

    4 publications · 111 citations

    4

    h-index

  5. Neural Information Processing Systems

    Conference · 3 publications · 345 citations

    3

    h-index

1–5