Olatunji Ruwase

Most-cited papers

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick +387 more
2022arXiv.org3,070 citations
Semantic Scholar
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Sam Adé Jacobs, A. Awan, J. Aneja +85 more
2024arXiv.org2,593 citations
Semantic Scholar
DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, Yuxiong He
2020Knowledge Discovery and Data Mining2,316 citations
Semantic Scholar
ZeRO: Memory optimizations Toward Training Trillion Parameter Models
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He
2019International Conference for High Performance Computing, Networking, Storage and Analysis2,174 citations Open access
Semantic Scholar
DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
Reza Yazdani Aminabadi, Samyam Rajbhandari, A. Awan, Cheng Li +7 more
2022International Conference for High Performance Computing, Networking, Storage and Analysis730 citations Open access
Semantic Scholar
ZeRO-Offload: Democratizing Billion-Scale Model Training
Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase +4 more
2021USENIX Annual Technical Conference671 citations
Semantic Scholar
ZeRO: Memory Optimization Towards Training A Trillion Parameter Models
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He
2019arXiv.org616 citations
Semantic Scholar
ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep learning
Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith +1 more
2021International Conference for High Performance Computing, Networking, Storage and Analysis588 citations
Semantic Scholar
A Practical Dynamic Buffer Overflow Detector
Olatunji Ruwase, Monica S. Lam
2004Network and Distributed System Security Symposium400 citations
Semantic Scholar
Accelerating Deep Convolutional Neural Networks Using Specialized Hardware
Kalin Ovtcharov, Olatunji Ruwase, Joo-Young Kim, J. Fowers +2 more
2015344 citations
Semantic Scholar
Flexible Hardware Acceleration for Instruction-Grain Program Monitoring
Shi-Min Chen, M. Kozuch, Theodoros Strigkos, B. Falsafi +6 more
2008International Symposium on Computer Architecture148 citations Open access
Semantic Scholar
DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
Z. Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari +15 more
2023arXiv.org134 citations Open access
Semantic Scholar
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
Zhenyu (Allen) Zhang, Runjin Chen, Shi-Wei Liu, Zhe-Wei Yao +4 more
2024Neural Information Processing Systems101 citations
Semantic Scholar
ZeRO++: Extremely Efficient Collective Communication for Giant Model Training
Guan-Hua Wang, He-Yang Qin, S. Jacobs, Connor Holmes +5 more
2023arXiv.org100 citations Open access
Semantic Scholar
Performance Modeling and Scalability Optimization of Distributed Deep Learning Systems
Feng Yan, Olatunji Ruwase, Yuxiong He, Trishul M. Chilimbi
2015Knowledge Discovery and Data Mining100 citations
Semantic Scholar
A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training
Siddharth Singh, Olatunji Ruwase, A. Awan, Samyam Rajbhandari +2 more
2023International Conference on Supercomputing90 citations Open access
Semantic Scholar
Parallelizing dynamic information flow tracking
Olatunji Ruwase, Phillip B. Gibbons, T. Mowry, V. Ramachandran +3 more
2008ACM Symposium on Parallelism in Algorithms and Architectures71 citations
Semantic Scholar
Ditto: a system for opportunistic caching in multi-hop wireless networks
Fahad R. Dogar, Amar Phanishayee, Himabindu Pucha, Olatunji Ruwase +1 more
2008ACM/IEEE International Conference on Mobile Computing and Networking61 citations
Semantic Scholar
Page overlays: An enhanced virtual memory framework to enable fine-grained memory management
Vivek Seshadri, Gennady Pekhimenko, Olatunji Ruwase, Onur Mutlu +4 more
2015International Symposium on Computer Architecture49 citations
Semantic Scholar
Toward accelerating deep learning at scale using specialized hardware in the datacenter
Kalin Ovtcharov, Olatunji Ruwase, Joo-Young Kim, J. Fowers +2 more
2015IEEE Hot Chips Symposium48 citations
Semantic Scholar
HyperDrive: exploring hyperparameters with POP scheduling
Jeff Rasley, Yuxiong He, Feng Yan, Olatunji Ruwase +1 more
2017International Middleware Conference42 citations
Semantic Scholar
SERF: Efficient Scheduling for Fast Deep Neural Network Serving via Judicious Parallelism
Feng Yan, Olatunji Ruwase, Yuxiong He, E. Smirni
2016International Conference for High Performance Computing, Networking, Storage and Analysis32 citations
Semantic Scholar
ZeRO++: Extremely Efficient Collective Communication for Large Model Training
Guan-Hua Wang, He-Yang Qin, Sam Adé Jacobs, Xiaoxia Wu +7 more
2024International Conference on Learning Representations30 citations
Semantic Scholar
Optimizing CNNs on Multicores for Scalability, Performance and Goodput
Samyam Rajbhandari, Yuxiong He, Olatunji Ruwase, Michael Carbin +1 more
2017International Conference on Architectural Support for Programming Languages and Operating Systems29 citations
Semantic Scholar
FastPersist: Accelerating Model Checkpointing in Deep Learning
Guan-Hua Wang, Olatunji Ruwase, Bing Xie, Yuxiong He
2024arXiv.org27 citations
Semantic Scholar
Quant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric Algorithm-System Co-Design on Modern GPUs
Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shi-Yang Chen +9 more
2024USENIX Annual Technical Conference27 citations
Semantic Scholar
Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping
Guan-Hua Wang, Cheng-Ming Zhang, Zhe-Yu Shen, Ang Li +1 more
2024arXiv.org26 citations
Semantic Scholar
Decoupled lifeguards: enabling path optimizations for dynamic correctness checking tools
Olatunji Ruwase, Shi-Min Chen, Phillip B. Gibbons, T. Mowry
2010ACM-SIGPLAN Symposium on Programming Language Design and Implementation26 citations
Semantic Scholar
FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design
Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shi-Yang Chen +9 more
2024arXiv.org24 citations
Semantic Scholar
Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelism
Xinyu Lian, Sam Adé Jacobs, Lev Kurilenko, Masahiro Tanaka +3 more
2024USENIX Annual Technical Conference23 citations
Semantic Scholar
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks
Xiaoxia Wu, Haojun Xia, Stephen Youn, Zhen Zheng +8 more
2023arXiv.org21 citations
Semantic Scholar
Efficient Deep Neural Network Serving: Fast and Furious
Feng Yan, Yuxiong He, Olatunji Ruwase, E. Smirni
2018IEEE Transactions on Network and Service Management20 citations Open access
Semantic Scholar
Universal Checkpointing: Efficient and Flexible Checkpointing for Large Scale Distributed Training
Xinyu Lian, Sam Adé Jacobs, Lev Kurilenko, Masahiro Tanaka +3 more
2024arXiv.org18 citations
Semantic Scholar
Flexible Hardware Acceleration for Instruction-Grain Lifeguards
Shi-Min Chen, M. Kozuch, Phillip B. Gibbons, Michael P. Ryan +6 more
2009IEEE Micro17 citations Open access
Semantic Scholar
LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory
Reza Yazdani, Olatunji Ruwase, Minjia Zhang, Yuxiong He +2 more
2019arXiv.org16 citations
Semantic Scholar
Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer
Jinghan Yao, Sam Adé Jacobs, Masahiro Tanaka, Olatunji Ruwase +3 more
2024Conference on Machine Learning and Systems13 citations
Semantic Scholar
Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft
Minjia Zhang, Samyam Rajbhandari, Wenhan Wang, Elton Zheng +5 more
2019USENIX Conference on Operational Machine Learning13 citations
Semantic Scholar
SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement
He-Yang Qin, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan +2 more
2021Neural Information Processing Systems12 citations
Semantic Scholar
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention
Z. Yao, Xiaoxia Wu, Cong-Long Li, Minjia Zhang +5 more
2023arXiv.org11 citations Open access
Semantic Scholar
SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips
Xinyu Lian, Masahiro Tanaka, Olatunji Ruwase, Min-Jia Zhang
2025International Conference on Architectural Support for Programming Languages and Operating Systems10 citations Open access
Semantic Scholar
SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks
Reza Yazdani Aminabadi, Olatunji Ruwase, Minjia Zhang, Yuxiong He +2 more
2019ACM Transactions on Embedded Computing Systems10 citations Open access
Semantic Scholar
Mojito: Motion Trajectory and Intensity Control for Video Generation
Xuehai He, Shuohang Wang, Jianwei Yang, Xiaoxia Wu +6 more
2024arXiv.org9 citations
Semantic Scholar
ZenFlow: Enabling Stall-Free Offloading for LLM Training
Ting-Feng Lan, Yu-Sen Wu, Bin Ma, Zhao-Yuan Su +6 more
2025Proceedings of the ACM on Management of Data8 citations Open access
Semantic Scholar
Guardrail: a high fidelity approach to protecting hardware devices from buggy drivers
Olatunji Ruwase, M. Kozuch, Phillip B. Gibbons, T. Mowry
2014International Conference on Architectural Support for Programming Languages and Operating Systems8 citations Open access
Semantic Scholar
RecFlex: Enabling Feature Heterogeneity-Aware Optimization for Deep Recommendation Models with Flexible Schedules
Zaifeng Pan, Zhen Zheng, Feng Zhang, Bing Xie +6 more
2024International Conference for High Performance Computing, Networking, Storage and Analysis5 citations
Semantic Scholar
DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training
Masahiro Tanaka, Duo Li, U. Chand, Ali Zafar +2 more
2025arXiv.org3 citations
Semantic Scholar
MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
Zhao-Yuan Su, Olatunji Ruwase, Karthik Ganesan, Aurick Qiao +4 more
2026arXiv.org2 citations
Semantic Scholar
AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism
Ahan Gupta, Zhihao Wang, Neel Dani, Masahiro Tanaka +2 more
2026arXiv.org1 citations
Semantic Scholar
Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models
Xinyu Lian, Walid Krichene, Beichen Huang, Masahiro Tanaka +3 more
2026arXiv.org1 citations
Semantic Scholar
Toward Accelerating Deep Learning at Scale Using Specialized Logic
Kalin Ovtcharov, Olatunji Ruwase, Joo-Young Kim, J. Fowers +2 more
20151 citations
Semantic Scholar