Concepedia

Gradient Accumulation

Gradient Accumulation is a computational technique in deep learning optimization whereby gradients are accumulated over multiple mini-batches before performing a single parameter update, enabling effective training with larger batch sizes under limited memory constraints.

23

Publications

3.6K

Citations

102

Authors

46

Institutions

Publications per year

2017–2026

16

Parents

Publications

23

BinaryConnect: Training Deep Neural Networks with binary weights during propagations

Matthieu Courbariaux, Yoshua Bengio, Jean‐Pierre David · PolyPublie (École Polytechnique de Montréal) · 2015 · 1.8K citations · Full text

, , +13

Mixed Precision Training

Paulius Micikevicius, Sharan Narang, Jonah Alben et al. · arXiv (Cornell University) · 2017 · 881 citations · Full text

, , +15

Mixed Precision Training

Paulius Micikevicius, Sharan Narang, Jonah Alben et al. · arXiv (Cornell University) · 2017 · 203 citations · Full text

, , +15

Toward Communication-Efficient Federated Learning in the Internet of Things With Edge Computing

Haifeng Sun, F. Richard Yu, Qi Qi et al. · IEEE Internet of Things Journal · 2020 · 91 citations

, , +12

Performance Evaluation of Leader–Follower-Based Mobile Molecular Communication Networks for Target Detection Applications

Tadashi Nakano, Yutaka Okaie, Shouhei Kobayashi et al. · IEEE Transactions on Communications · 2016 · 81 citations

, , +12

1–5 of 23

Authors

102

  1. PM
    Paulius Micikevicius

    Nvidia (United Kingdom), United Kingdom

    2 publications · 1.1K citations

    2

    h-index

  2. JL
    Jiadong Lin

    First Affiliated Hospital of Xi'an Jiaotong University, China

    2 publications · 50 citations

    2

    h-index

  3. HW
    Hao Wu

    Xi'an Jiaotong University, China

    2 publications · 1.1K citations

    2

    h-index

  4. EE
    Erich Elsen

    Google (United States), United States

    2 publications · 1.1K citations

    2

    h-index

  5. OK
    Oleksii Kuchaiev

    University of California, Irvine, United States

    2 publications · 1.1K citations

    2

    h-index

1–5 of 102

Institutions

46

  1. Beijing University of Posts and Telecommunications

    Beijing, China

    2 h-index

    10 publications

    705 citations

  2. Carleton University

    Ottawa, Canada

    2 h-index

    2 publications

    141 citations

  3. Tsinghua University

    Beijing, China

    2 h-index

    9 publications

    247 citations

  4. Baidu (China)

    Beijing, China

    1 h-index

    2 publications

    406 citations

  5. École Nationale Supérieure des Mines de Paris

    Paris, France

    1 h-index

    2 publications

    52 citations

  6. Université de Montréal

    Montreal, Canada

    1 h-index

    1 publications

    1.8K citations

  7. Brunel University of London

    London, United Kingdom

    1 h-index

    1 publications

    11 citations

  8. Euphrosyne Polotskaya State University

    Navapolatsk, Belarus

    1 h-index

    2 publications

    22 citations

1–8 of 46

Venues

  1. arXiv (Cornell University)

    Repository · 5 publications · 1.2K citations

    5

    h-index

  2. 1

    h-index

  3. Cement and Concrete Composites

    1 publications · 30 citations

    1

    h-index

  4. IEEE Transactions on Communications

    1 publications · 81 citations

    1

    h-index

  5. IEEE Internet of Things Journal

    1 publications · 91 citations

    1

    h-index

1–5