Reza Yazdani Aminabadi

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Shaden Smith, Mostofa Patwary, Brandon Norick et al. · arXiv (Cornell University) · 2022 · 299 citations · Full text

Computing, Computer Science, High-performance Computing +17

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan et al. · 2022 · 225 citations

Computing, Computer Engineering, Computer Science +17

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang et al. · arXiv (Cornell University) · 2022 · 72 citations · Full text

Computing, Computer Engineering, Computer Science +16

ZeRO-Offload: Democratizing Billion-Scale Model Training

Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi et al. · arXiv (Cornell University) · 2021 · 61 citations · Full text

Computing, Computer Science, High-performance Computing +17

DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale

Samyam Rajbhandari, Conglong Li, Zhewei Yao et al. · arXiv (Cornell University) · 2022 · 55 citations · Full text

Computing, Computer Engineering, Computer Science +17