Shu-Ming Ma

Most-cited papers

Kosmos-2: Grounding Multimodal Large Language Models to the World
Zhiliang Peng, Wen-Hui Wang, Li Dong, Y. Hao +4 more
2023International Conference on Learning Representations1,371 citations Open access
Semantic Scholar
A whole-slide foundation model for digital pathology from real-world data
Han-Wen Xu, N. Usuyama, Jaspreet Bagga, Sheng Zhang +24 more
2024Nature1,208 citations Open access
Semantic Scholar
Retentive Network: A Successor to Transformer for Large Language Models
Yutao Sun, Li Dong, Shao-Han Huang, Shu-Ming Ma +4 more
2023arXiv.org823 citations
Semantic Scholar
Language Is Not All You Need: Aligning Perception with Language Models
Shao-Han Huang, Li Dong, Wen-Hui Wang, Y. Hao +14 more
2023Neural Information Processing Systems820 citations Open access
Semantic Scholar
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Shu-Ming Ma, Hongyu Wang, Ling-Xiao Ma, Lei Wang +6 more
2024arXiv.org502 citations
Semantic Scholar
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers
Damai Dai, Yutao Sun, Li Dong, Y. Hao +3 more
2023Annual Meeting of the Association for Computational Linguistics425 citations Open access
Semantic Scholar
LongNet: Scaling Transformers to 1, 000, 000, 000 Tokens
Jiayu Ding, Shu-Ming Ma, Li Dong, Xing-Xing Zhang +4 more
2023arXiv.org291 citations
Semantic Scholar
BitNet: Scaling 1-bit Transformers for Large Language Models
Hongyu Wang, Shu-Ming Ma, Li Dong, Shao-Han Huang +6 more
2023arXiv.org285 citations
Semantic Scholar
DeepNet: Scaling Transformers to 1,000 Layers
Hongyu Wang, Shu-Ming Ma, Li Dong, Shao-Han Huang +2 more
2022IEEE Transactions on Pattern Analysis and Machine Intelligence284 citations Open access
Semantic Scholar
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
Damai Dai, Yutao Sun, Li Dong, Y. Hao +3 more
2022267 citations
Semantic Scholar
On the Representation Collapse of Sparse Mixture of Experts
Zewen Chi, Li Dong, Shao-Han Huang, Damai Dai +8 more
2022Neural Information Processing Systems227 citations Open access
Semantic Scholar
A Length-Extrapolatable Transformer
Yutao Sun, Li Dong, Barun Patra, Shu-Ming Ma +5 more
2022Annual Meeting of the Association for Computational Linguistics192 citations Open access
Semantic Scholar
You Only Cache Once: Decoder-Decoder Architectures for Language Models
Yutao Sun, Li Dong, Yi Zhu, Shao-Han Huang +5 more
2024Neural Information Processing Systems176 citations
Semantic Scholar
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
Wenkai Yang, Shu-Ming Ma, Yankai Lin, F. Wei
2025Neural Information Processing Systems144 citations
Semantic Scholar
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
Zewen Chi, Shao-Han Huang, Li Dong, Shu-Ming Ma +7 more
2021Annual Meeting of the Association for Computational Linguistics141 citations Open access
Semantic Scholar
StableMoE: Stable Routing Strategy for Mixture of Experts
Damai Dai, Li Dong, Shu-Ming Ma, Bo Zheng +3 more
2022Annual Meeting of the Association for Computational Linguistics136 citations Open access
Semantic Scholar
Kosmos-2.5: A Multimodal Literate Model
Teng-Chao Lv, Yu-Pan Huang, Jing-Ye Chen, Yu-Zhong Zhao +12 more
2023arXiv.org122 citations Open access
Semantic Scholar
Language Models are General-Purpose Interfaces
Y. Hao, Haoyu Song, Li Dong, Shao-Han Huang +4 more
2022arXiv.org117 citations Open access
Semantic Scholar
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
Shu-Ming Ma, Li Dong, Shao-Han Huang, Dong-Dong Zhang +5 more
2021arXiv.org93 citations
Semantic Scholar
mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Zewen Chi, Li Dong, Shu-Ming Ma, S. Mao +3 more
2021Conference on Empirical Methods in Natural Language Processing82 citations Open access
Semantic Scholar
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation
Y. Jiang, Tianyu Liu, Shu-Ming Ma, Dong-Dong Zhang +6 more
2021North American Chapter of the Association for Computational Linguistics82 citations Open access
Semantic Scholar
BitNet b1.58 2B4T Technical Report
Shu-Ming Ma, Hongyu Wang, Shao-Han Huang, Xingxing Zhang +4 more
2025arXiv.org52 citations
Semantic Scholar
Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders
Guanhua Chen, Shu-Ming Ma, Yun Chen, Li Dong +4 more
2021Conference on Empirical Methods in Natural Language Processing46 citations Open access
Semantic Scholar
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
Jian Yang, Shu-Ming Ma, Hao-Yang Huang, Dong-Dong Zhang +7 more
2021Conference on Machine Translation46 citations
Semantic Scholar
Multi-Head Mixture-of-Experts
Xun Wu, Shao-Han Huang, Wen-Hui Wang, Shu-Ming Ma +2 more
2024Neural Information Processing Systems44 citations
Semantic Scholar
Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt
Lianzhe Huang, Shu-Ming Ma, Dong-Dong Zhang, Furu Wei +1 more
2022Conference on Empirical Methods in Natural Language Processing39 citations Open access
Semantic Scholar
Bitnet.cpp: Efficient Edge Inference for Ternary LLMs
Jinheng Wang, Hansong Zhou, Ting Song, Shi-Jie Cao +6 more
2025Annual Meeting of the Association for Computational Linguistics37 citations
Semantic Scholar
A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model
Xin Sun, Tao Ge, Shu-Ming Ma, Jingjing Li +2 more
2022International Joint Conference on Artificial Intelligence34 citations Open access
Semantic Scholar
CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation
Jian Yang, Jian Yang, Shao-Han Huang, Shu-Ming Ma +6 more
2022Conference on Empirical Methods in Natural Language Processing34 citations Open access
Semantic Scholar
Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
Hongyu Wang, Shu-Ming Ma, Rui-Ping Wang, F. Wei
2024arXiv.org33 citations
Semantic Scholar
Multilingual Agreement for Multilingual Neural Machine Translation
Jian Yang, Yuwei Yin, Shu-Ming Ma, Hao-Yang Huang +3 more
2021Annual Meeting of the Association for Computational Linguistics32 citations Open access
Semantic Scholar
Next Block Prediction: Video Generation via Semi-Autoregressive Modeling
Shuhuai Ren, Shu-Ming Ma, Xu Sun, F. Wei
2025arXiv.org30 citations
Semantic Scholar
Foundation Transformers
Hongyu Wang, Shu-Ming Ma, Shao-Han Huang, Li Dong +11 more
2022arXiv.org30 citations Open access
Semantic Scholar
When an Image is Worth 1, 024 x 1, 024 Words: A Case Study in Computational Pathology
Wen-Hui Wang, Shu-Ming Ma, Han-Wen Xu, N. Usuyama +3 more
2023arXiv.org28 citations
Semantic Scholar
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator
Jian Yang, Shu-Ming Ma, Li Dong, Shao-Han Huang +6 more
2022Annual Meeting of the Association for Computational Linguistics28 citations Open access
Semantic Scholar
BitNet: 1-bit Pre-training for Large Language Models
Hongyu Wang, Shu-Ming Ma, Ling-Xiao Ma, Lei Wang +8 more
2025Journal of machine learning research25 citations
Semantic Scholar
GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation
Jian Yang, Jian Yang, Yuwei Yin, Shu-Ming Ma +4 more
2022IEEE/ACM Transactions on Audio Speech and Language Processing25 citations Open access
Semantic Scholar
Auto-ICL: In-Context Learning without Human Supervision
Jinghan Yang, Shu-Ming Ma, Fu-Ru Wei
2023arXiv.org24 citations
Semantic Scholar
BitNet a4.8: 4-bit Activations for 1-bit LLMs
Hongyu Wang, Shu-Ming Ma, F. Wei
2024arXiv.org22 citations
Semantic Scholar
Are More Layers Beneficial to Graph Transformers?
Haiteng Zhao, Shu-Ming Ma, Dong-Dong Zhang, Zhi-Hong Deng +1 more
2023International Conference on Learning Representations22 citations Open access
Semantic Scholar
BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
Hongyu Wang, Shu-Ming Ma, F. Wei
2025arXiv.org20 citations
Semantic Scholar
How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?
Weijia Xu, Shu-Ming Ma, Dong-Dong Zhang, Marine Carpuat
2021Findings20 citations Open access
Semantic Scholar
Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation
Guanhua Chen, Shu-Ming Ma, Yun Chen, Dong-Dong Zhang +3 more
2021Annual Meeting of the Association for Computational Linguistics19 citations Open access
Semantic Scholar
1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
Jinheng Wang, Hansong Zhou, Ting Song, Shaoguang Mao +4 more
2024arXiv.org18 citations
Semantic Scholar
Discourse-Centric Evaluation of Document-level Machine Translation with a New Densely Annotated Parallel Corpus of Novels
Y. Jiang, Tianyu Liu, Shu-Ming Ma, Dong-Dong Zhang +2 more
2023Annual Meeting of the Association for Computational Linguistics17 citations Open access
Semantic Scholar
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Y. Jiang, Tianyu Liu, Shu-Ming Ma, Dong-Dong Zhang +2 more
2023arXiv.org14 citations Open access
Semantic Scholar
Magneto: A Foundation Transformer
Hongyu Wang, Shu-Ming Ma, Shao-Han Huang, Li Dong +11 more
2023International Conference on Machine Learning13 citations
Semantic Scholar
TorchScale: Transformers at Scale
Shu-Ming Ma, Hongyu Wang, Shao-Han Huang, Wen-Hui Wang +7 more
2022arXiv.org12 citations
Semantic Scholar
UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation
Jian Yang, Jian Yang, Yuwei Yin, Shu-Ming Ma +5 more
2022International Joint Conference on Artificial Intelligence11 citations Open access
Semantic Scholar
Improving Multilingual Neural Machine Translation with Auxiliary Source Languages
Weijia Xu, Yuwei Yin, Shu-Ming Ma, Dong-Dong Zhang +1 more
2021Conference on Empirical Methods in Natural Language Processing11 citations Open access
Semantic Scholar