Concepedia

Weight Decay

Weight Decay is a regularization technique in machine learning that belongs to the class of parameter norm penalties, specifically distinguished by its implementation of L2 penalty on the model weights to prevent overfitting by discouraging excessively large weight values.

258

Publications

37.6K

Citations

669

Authors

277

Institutions

Publications per year

2017–2026

145

Parents

Publications

258

Decoupled Weight Decay Regularization

Ilya Loshchilov, Frank Hutter · arXiv (Cornell University) · 2017 · 9.1K citations · Full text

, , +16

A Practical Bayesian Framework for Backpropagation Networks

David Mackay · Neural Computation · 1992 · 2.9K citations

, , +16

Best practices for convolutional neural networks applied to visual document analysis

P. Simard, Don Steinkraus, John Platt · 2005 · 2.8K citations

, , +10

On Calibration of Modern Neural Networks

Chuan Guo, Geoff Pleiss, Yu Sun et al. · arXiv (Cornell University) · 2017 · 1.7K citations · Full text

, , +16

Second order derivatives for network pruning: Optimal Brain Surgeon

Babak Hassibi, David G. Stork · CaltechAUTHORS (California Institute of Technology) · 1992 · 1.5K citations · Full text

, , +11

1–5 of 258

Authors

669

  1. CL
    Chi-Sing Leung

    City University of Hong Kong, Hong Kong SAR China

    7 publications · 273 citations

    7

    h-index

  2. JS
    John Sum

    City University of Hong Kong, Hong Kong SAR China

    5 publications · 161 citations

    5

    h-index

  3. RP
    Rahul Parhi

    University of Wisconsin–Madison, United States

    5 publications · 253 citations

    5

    h-index

  4. RD
    Robert D. Nowak

    University of Wisconsin–Madison, United States

    5 publications · 253 citations

    5

    h-index

  5. SA
    Sanjeev Arora

    Princeton University, United States

    3 publications · 69 citations

    4

    h-index

1–5 of 669

Institutions

277

  1. Google (United States)

    Mountain View, United States

    8 h-index

    19 publications

    1.8K citations

  2. City University of Hong Kong

    Hong Kong, Hong Kong

    7 h-index

    12 publications

    498 citations

  3. University of Wisconsin–Madison

    Madison, United States

    6 h-index

    13 publications

    535 citations

  4. University of California, Los Angeles

    Los Angeles, United States

    6 h-index

    13 publications

    476 citations

  5. Princeton University

    Princeton, United States

    6 h-index

    11 publications

    1.3K citations

  6. Hefei University of Technology

    Hefei, China

    5 h-index

    7 publications

    433 citations

  7. Cornell University

    Ithaca, United States

    5 h-index

    12 publications

    11.1K citations

  8. National Chung Hsing University

    Taichung, Taiwan

    4 h-index

    4 publications

    125 citations

1–8 of 277

Venues

  1. arXiv (Cornell University)

    Repository · 51 publications · 14.3K citations

    26

    h-index

  2. IEEE Transactions on Neural Networks

    13 publications · 831 citations

    13

    h-index

  3. Neural Computation

    9 publications · 4.9K citations

    9

    h-index

  4. 6

    h-index

  5. Journal of Physics A Mathematical and General

    5 publications · 197 citations

    5

    h-index

1–5