Samyam Rajbhandari

ZeRO: Memory optimizations Toward Training Trillion Parameter Models

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase et al. · 2020 · 672 citations

Engineering, Machine Learning, Model Tuning +16

DeepSpeed

Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase et al. · 2020 · 653 citations

Large Language Models, Natural Language Processing, Large Ai Model +15

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Shaden Smith, Mostofa Patwary, Brandon Norick et al. · arXiv (Cornell University) · 2022 · 299 citations · Full text

Artificial Intelligence, Llm Fine-tuning, Megatron-turing Nlg 530B +21

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan et al. · 2022 · 198 citations

Convolutional Neural Network, Engineering, Machine Learning +19

ZeRO-infinity

Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley et al. · 2021 · 193 citations

Cluster Computing, Gpu Architecture, Engineering +15