Cong-Long Li

Most-cited papers

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick +387 more
2022arXiv.org3,070 citations
Semantic Scholar
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers
Z. Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu +2 more
2022Neural Information Processing Systems834 citations Open access
Semantic Scholar
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
Samyam Rajbhandari, Cong-Long Li, Z. Yao, Min-Jia Zhang +4 more
2022International Conference on Machine Learning688 citations
Semantic Scholar
OpenFold: retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization
Gustaf Ahdritz, N. Bouatta, C. Floristean, Sachin Kadyan +30 more
2023bioRxiv489 citations Open access
Semantic Scholar
Scaling Video Analytics on Constrained Edge Nodes
Christopher Canel, Thomas Kim, Giulio Zhou, Cong-Long Li +4 more
2019USENIX workshop on Tackling computer systems problems with machine learning techniques195 citations
Semantic Scholar
Scheduling techniques for hybrid circuit/packet networks
He Liu, Matthew K. Mukerjee, Cong-Long Li, Nicolas Feltman +8 more
2015Conference on Emerging Network Experiment and Technology159 citations Open access
Semantic Scholar
Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination
Cong-Long Li, Minjia Zhang, D. Andersen, Yu-Xiong He
2020SIGMOD Conference135 citations Open access
Semantic Scholar
DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
Z. Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari +15 more
2023arXiv.org134 citations Open access
Semantic Scholar
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
Han-Lin Tang, Shaoduo Gan, A. Awan, Samyam Rajbhandari +5 more
2021International Conference on Machine Learning120 citations
Semantic Scholar
GD-Wheel: a cost-aware replacement policy for key-value stores
Cong-Long Li, A. Cox
2015European Conference on Computer Systems70 citations
Semantic Scholar
The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models
Cong-Long Li, Minjia Zhang, Yu-Xiong He
2021Neural Information Processing Systems59 citations Open access
Semantic Scholar
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
Akiko Aizawa, Eiji Aramaki, Bo-Wen Chen, Fei Cheng +76 more
2024arXiv.org46 citations
Semantic Scholar
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing
Cong-Long Li, Z. Yao, Xiaoxia Wu, Minjia Zhang +3 more
2022AAAI Conference on Artificial Intelligence45 citations Open access
Semantic Scholar
1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB’s Convergence Speed
Cong-Long Li, A. Awan, Hanlin Tang, Samyam Rajbhandari +1 more
2021International Conference on High Performance Computing37 citations
Semantic Scholar
Curriculum Learning: A Regularization Method for Efficient and Stable Billion-Scale GPT Model Pre-Training
Cong-Long Li, Min-Jia Zhang, Yu-Xiong He
2021arXiv.org33 citations
Semantic Scholar
Extreme Compression for Pre-trained Transformers Made Simple and Efficient
Xiaoxia Wu, Z. Yao, Minjia Zhang, Cong-Long Li +1 more
2022Neural Information Processing Systems31 citations Open access
Semantic Scholar
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
S. Song, Bonnie Kruft, Min-Jia Zhang, Cong-Long Li +88 more
2023arXiv.org27 citations Open access
Semantic Scholar
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam
Yu-Cheng Lu, Cong-Long Li, Min-Jia Zhang, C. De Sa +1 more
2022International Conference on Learning Representations23 citations
Semantic Scholar
Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers
Z. Yao, Xiaoxia Wu, Cong-Long Li, Connor Holmes +3 more
2022arXiv.org16 citations Open access
Semantic Scholar
Using Indirect Routing to Recover from Network Traffic Scheduling Estimation Error
Cong-Long Li, Matthew K. Mukerjee, D. Andersen, S. Seshan +3 more
2017Symposium on Architectures for Networking and Communications Systems13 citations
Semantic Scholar
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention
Z. Yao, Xiaoxia Wu, Cong-Long Li, Minjia Zhang +5 more
2023arXiv.org11 citations Open access
Semantic Scholar
Better Caching in Search Advertising Systems with Rapid Refresh Predictions
Cong-Long Li, D. Andersen, Qiang Fu, S. Elnikety +1 more
2018The Web Conference11 citations Open access
Semantic Scholar
Workload analysis and caching strategies for search advertising systems
Cong-Long Li, D. Andersen, Qiang Fu, S. Elnikety +1 more
2017ACM Symposium on Cloud Computing9 citations
Semantic Scholar
Reducing DRAM row activations with eager read/write clustering
Myeongjae Jeon, Cong-Long Li, A. Cox, S. Rixner
2013ACM Transactions on Architecture and Code Optimization (TACO)6 citations Open access
Semantic Scholar
Picking Interesting Frames in Streaming Video
Christopher Canel, Thomas Kim, Giulio Zhou, Cong-Long Li +4 more
20185 citations
Semantic Scholar
XTC: Extreme Compression for Pre-Trained Transformers Made Simple and Efficient
Yu-Xiong He, Cong-Long Li, Xiaoxia Wu, Zhe-Wei Yao +1 more
2022Advances in Neural Information Processing Systems 351 citations
Semantic Scholar
Efficient Multi-Tenant Inference on Video using Microclassifiers
Giulio Zhou, Thomas Kim, Christopher Canel, Cong-Long Li +4 more
20180 citations
Semantic Scholar