Mostofa Patwary

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Mohammad Shoeybi, Mostofa Patwary, Raul Puri et al. · arXiv (Cornell University) · 2019 · 819 citations · Full text

Llm Fine-tuning, Engineering, Machine Learning +22

Scalable Bayesian Optimization Using Deep Neural Networks

Jasper Snoek, Oren Rippel, Kevin Swersky et al. · arXiv (Cornell University) · 2015 · 439 citations · Full text

Artificial Intelligence, Model Optimization, Hyperparameter Estimation +13

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Shaden Smith, Mostofa Patwary, Brandon Norick et al. · arXiv (Cornell University) · 2022 · 299 citations · Full text

Artificial Intelligence, Llm Fine-tuning, Megatron-turing Nlg 530B +21

BioMegatron: Larger Biomedical Domain Language Model

Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina et al. · 2020 · 114 citations · Full text

Engineering, Part-of-speech Tagging, Large Language Model +16

MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models

Peng Xu, Mostofa Patwary, Mohammad Shoeybi et al. · 2020 · 102 citations · Full text

Artificial Intelligence, Controllable Story Generation, Engineering +20