Reza Yazdani Aminabadi

Most-cited papers

1. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

   Shaden Smith, M. Patwary, Brandon Norick, P. LeGresley +16 more

   2022arXiv.org873 citations

   Semantic Scholar

   Save

2. ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

   Z. Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu +2 more

   2022Neural Information Processing Systems834 citations Open access

   Semantic Scholar

   Save

3. DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

   Reza Yazdani Aminabadi, Samyam Rajbhandari, A. Awan, Cheng Li +7 more

   2022International Conference for High Performance Computing, Networking, Storage and Analysis730 citations Open access

   Semantic Scholar

   Save

4. DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale

   Samyam Rajbhandari, Cong-Long Li, Z. Yao, Min-Jia Zhang +4 more

   2022International Conference on Machine Learning688 citations

   Semantic Scholar

   Save

5. ZeRO-Offload: Democratizing Billion-Scale Model Training

   Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase +4 more

   2021USENIX Annual Technical Conference671 citations

   Semantic Scholar

   Save

6. DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

   Connor Holmes, Masahiro Tanaka, Michael Wyatt, A. Awan +7 more

   2024arXiv.org171 citations

   Semantic Scholar

   Save

7. DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales

   Z. Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari +15 more

   2023arXiv.org134 citations Open access

   Semantic Scholar

   Save

8. System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models

   Sam Adé Jacobs, Masahiro Tanaka, Cheng-Ming Zhang, Min-Jia Zhang +4 more

   2024IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum93 citations

   Semantic Scholar

   Save

9. Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases

   Xiaoxia Wu, Cheng Li, Reza Yazdani Aminabadi, Z. Yao +1 more

   2023International Conference on Machine Learning45 citations

   Semantic Scholar

   Save

10. Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases

    Xiaoxia Wu, Cheng Li, Reza Yazdani Aminabadi, Z. Yao +1 more

    2023arXiv.org33 citations Open access

    Semantic Scholar

    Save

11. ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks

    Xiaoxia Wu, Haojun Xia, Stephen Youn, Zhen Zheng +8 more

    2023arXiv.org21 citations

    Semantic Scholar

    Save

12. SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks

    Reza Yazdani Aminabadi, Olatunji Ruwase, Minjia Zhang, Yuxiong He +2 more

    2019ACM Transactions on Embedded Computing Systems10 citations Open access

    Semantic Scholar

    Save

13. Fault-tolerant 3-D network-on-chip design using dynamic link sharing

    Seyyed Hossein Seyyedaghaei Rezaei, Mehdi Modarressi, Reza Yazdani Aminabadi, M. Daneshtalab

    2016Design, Automation and Test in Europe6 citations

    Semantic Scholar

    Save

14. ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers

    Zhe-Wei Yao, Reza Yazdani Aminabadi, Stephen Youn, Xiaoxia Wu +2 more

    2023arXiv.org2 citations

    Semantic Scholar

    Save

15. Designing Draft Models for Speculative Decoding

    Reza Yazdani Aminabadi, Samyam Rajbhandari, Ahmad Awan, Cheng Li +34 more

    2 citations

    Semantic Scholar

    Save