Yuxiong He

ZeRO: Memory optimizations Toward Training Trillion Parameter Models

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase et al. · 2020 · 672 citations

Engineering, Machine Learning, Model Tuning +16

DeepSpeed

Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase et al. · 2020 · 653 citations

Large Language Models, Natural Language Processing, Large Ai Model +15

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Shaden Smith, Mostofa Patwary, Brandon Norick et al. · arXiv (Cornell University) · 2022 · 299 citations · Full text

Artificial Intelligence, Llm Fine-tuning, Megatron-turing Nlg 530B +21

OpenFold: retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

Gustaf Ahdritz, Nazim Bouatta, Christina Floristean et al. · Nature Methods · 2024 · 235 citations · Full text

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan et al. · 2022 · 198 citations

Convolutional Neural Network, Engineering, Machine Learning +19