Pengcheng He

Most-cited papers

1. DeBERTa: Decoding-enhanced BERT with Disentangled Attention

   Pengcheng He, Xiao-Dong Liu, Jianfeng Gao, Wei-Zhu Chen

   2020International Conference on Learning Representations4,263 citations

   Semantic Scholar

   Save

2. On the Variance of the Adaptive Learning Rate and Beyond

   Liyuan Liu, Haoming Jiang, Pengcheng He, Wei-Zhu Chen +3 more

   2019International Conference on Learning Representations2,325 citations

   Semantic Scholar

   Save

3. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

   Pengcheng He, Jian-Feng Gao, Wei-Zhu Chen

   2021International Conference on Learning Representations2,321 citations

   Semantic Scholar

   Save

4. Multi-Task Deep Neural Networks for Natural Language Understanding

   Xiaodong Liu, Pengcheng He, Wei-Zhu Chen, Jianfeng Gao

   2019Annual Meeting of the Association for Computational Linguistics1,378 citations Open access

   Semantic Scholar

   Save

5. Instruction Tuning with GPT-4

   Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley +1 more

   2023arXiv.org876 citations

   Semantic Scholar

   Save

6. Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning

   Qingru Zhang, Min-Shuo Chen, A. Bukharin, Pengcheng He +3 more

   2023International Conference on Learning Representations819 citations Open access

   Semantic Scholar

   Save

7. SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

   Haoming Jiang, Pengcheng He, Wei-Zhu Chen, Xiaodong Liu +2 more

   2019Annual Meeting of the Association for Computational Linguistics622 citations Open access

   Semantic Scholar

   Save

8. Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

   Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng +7 more

   2023arXiv.org556 citations Open access

   Semantic Scholar

   Save

9. AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning

   Qingru Zhang, Min-Shuo Chen, A. Bukharin, Nikos Karampatziakis +4 more

   2023492 citations

   Semantic Scholar

   Save

10. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models

    Yung-Sung Chuang, Yujia Xie, Hong-Yin Luo, Yoon Kim +2 more

    2023International Conference on Learning Representations485 citations Open access

    Semantic Scholar

    Save

11. Diffusion-GAN: Training GANs with Diffusion

    Zhen-Dong Wang, Huangjie Zheng, Pengcheng He, Wei-Zhu Chen +1 more

    2022International Conference on Learning Representations354 citations Open access

    Semantic Scholar

    Save

12. Query Rewriting for Retrieval-Augmented Large Language Models

    Xin-Bei Ma, Yeyun Gong, Pengcheng He, Hai Zhao +1 more

    2023arXiv.org320 citations Open access

    Semantic Scholar

    Save

13. LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models

    Yi-Xiao Li, Yifan Yu, Chen Liang, Pengcheng He +3 more

    2023International Conference on Learning Representations263 citations Open access

    Semantic Scholar

    Save

14. Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models

    Wei-Zhu Chen, Pengcheng He, Yi-Fan Jiang, Pei-Hao Wang +4 more

    2023Neural Information Processing Systems209 citations Open access

    Semantic Scholar

    Save

15. Adversarial Training for Large Neural Language Models

    Xiao-Dong Liu, Hao Cheng, Pengcheng He, Wei-Zhu Chen +3 more

    2020arXiv.org209 citations

    Semantic Scholar

    Save

16. Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding

    Xiaodong Liu, Pengcheng He, Wei-Zhu Chen, Jianfeng Gao

    2019arXiv.org197 citations

    Semantic Scholar

    Save

17. Guiding Large Language Models via Directional Stimulus Prompting

    Zekun Li, Baolin Peng, Pengcheng He, Michel Galley +2 more

    2023Neural Information Processing Systems166 citations Open access

    Semantic Scholar

    Save

18. LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation

    Yixiao Li, Yifan Yu, Qingru Zhang, Chen Liang +3 more

    2023International Conference on Machine Learning153 citations Open access

    Semantic Scholar

    Save

19. Less is More: Task-aware Layer-wise Distillation for Language Model Compression

    Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He +2 more

    2022arXiv.org142 citations Open access

    Semantic Scholar

    Save

20. In-Context Learning Unlocked for Diffusion Models

    Wei-Zhu Chen, Pengcheng He, Yi-Fan Jiang, Ya-Dong Lu +4 more

    2023Neural Information Processing Systems129 citations Open access

    Semantic Scholar

    Save

21. PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

    Qingru Zhang, Simiao Zuo, Chen Liang, A. Bukharin +3 more

    2022International Conference on Machine Learning118 citations Open access

    Semantic Scholar

    Save

22. X-SQL: reinforce schema representation with context

    Pengcheng He, Yi Mao, K. Chakrabarti, Wei-Zhu Chen

    2019arXiv.org106 citations

    Semantic Scholar

    Save

23. Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection

    Zekun Li, Baolin Peng, Pengcheng He, Xi-Feng Yan

    2023Conference on Empirical Methods in Natural Language Processing93 citations Open access

    Semantic Scholar

    Save

24. GODEL: Large-Scale Pre-Training for Goal-Directed Dialog

    Baolin Peng, Michel Galley, Pengcheng He, Chris Brockett +5 more

    2022arXiv.org80 citations Open access

    Semantic Scholar

    Save

25. Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

    Chen Liang, Simiao Zuo, Min-Shuo Chen, Haoming Jiang +4 more

    2021Annual Meeting of the Association for Computational Linguistics78 citations Open access

    Semantic Scholar

    Save

26. Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning

    Tao Shen, Yi Mao, Pengcheng He, Guodong Long +2 more

    2020Conference on Empirical Methods in Natural Language Processing74 citations Open access

    Semantic Scholar

    Save

27. NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

    Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen +49 more

    2021Neural Information Processing Systems69 citations

    Semantic Scholar

    Save

28. Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention

    Yi-Chong Xu, Chenguang Zhu, Shuo-Hang Wang, Si-Qi Sun +6 more

    2021International Joint Conference on Artificial Intelligence66 citations Open access

    Semantic Scholar

    Save

29. The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding

    Xiao-Dong Liu, Yu Wang, Jianshu Ji, Hao Cheng +7 more

    2020Annual Meeting of the Association for Computational Linguistics63 citations Open access

    Semantic Scholar

    Save

30. MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

    Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He +2 more

    2022North American Chapter of the Association for Computational Linguistics62 citations Open access

    Semantic Scholar

    Save

31. OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

    Zhengbao Jiang, Yi Mao, Pengcheng He, Graham Neubig +1 more

    2022North American Chapter of the Association for Computational Linguistics60 citations Open access

    Semantic Scholar

    Save

32. HyperTuning: Toward Adapting Large Language Models without Back-propagation

    Jason Phang, Yi Mao, Pengcheng He, Wei-Zhu Chen

    2022International Conference on Machine Learning60 citations Open access

    Semantic Scholar

    Save

33. Truncated Diffusion Probabilistic Models

    Huangjie Zheng, Pengcheng He, Wei-Zhu Chen, Ming-Yuan Zhou

    2022International Conference on Learning Representations58 citations

    Semantic Scholar

    Save

34. Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization

    Pengcheng He, Baolin Peng, Liyang Lu, Song Wang +10 more

    2022Annual Meeting of the Association for Computational Linguistics55 citations Open access

    Semantic Scholar

    Save

35. Summaries, Highlights, and Action Items: Design, Implementation and Evaluation of an LLM-powered Meeting Recap System

    Sumit Asthana, Sagi Hilleli, Pengcheng He, Aaron L Halfaker

    2023Proc. ACM Hum. Comput. Interact.53 citations Open access

    Semantic Scholar

    Save

36. POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained models

    Korawat Tanwisuth, Shujian Zhang, Huangjie Zheng, Pengcheng He +1 more

    2023International Conference on Machine Learning53 citations Open access

    Semantic Scholar

    Save

37. ALLSH: Active Learning Guided by Local Sensitivity and Hardness

    Shujian Zhang, Chengyue Gong, Xing-Chao Liu, Pengcheng He +2 more

    2022NAACL-HLT31 citations Open access

    Semantic Scholar

    Save

38. A Hybrid Neural Network Model for Commonsense Reasoning

    Pengcheng He, Xiaodong Liu, Wei-Zhu Chen, Jianfeng Gao

    2019Conference on Empirical Methods in Natural Language Processing30 citations Open access

    Semantic Scholar

    Save

39. Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective

    Ming Zhong, Chenxin An, Weizhu Chen, Jiawei Han +1 more

    2023International Conference on Learning Representations21 citations

    Semantic Scholar

    Save

40. No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

    Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He +4 more

    2022International Conference on Learning Representations18 citations

    Semantic Scholar

    Save

41. Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPs

    Huangjie Zheng, Pengcheng He, Wei-Zhu Chen, Ming-Yuan Zhou

    2022arXiv.org16 citations

    Semantic Scholar

    Save

42. LMGQS: A Large-scale Dataset for Query-focused Summarization

    Ruochen Xu, Song Wang, Yang Liu, Shuo Wang +5 more

    2023Conference on Empirical Methods in Natural Language Processing11 citations Open access

    Semantic Scholar

    Save

43. Momentum Calibration for Text Generation

    Xingxing Zhang, Yi-Ran Liu, Xun Wang, Pengcheng He +4 more

    2022arXiv.org11 citations Open access

    Semantic Scholar

    Save

44. DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization

    Yu Li, Baolin Peng, Pengcheng He, Michel Galley +2 more

    2022Annual Meeting of the Association for Computational Linguistics10 citations Open access

    Semantic Scholar

    Save

45. Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach

    Simiao Zuo, Chen Liang, Haoming Jiang, Xiao-Dong Liu +4 more

    2021Conference on Empirical Methods in Natural Language Processing10 citations Open access

    Semantic Scholar

    Save

46. Proceedings of the 1st Workshop on NLP for Positive Impact

    J. Lee, Bai-Kun Liang, Haley Fong, Tian-Yi Zhang +496 more

    20219 citations Open access

    Semantic Scholar

    Save

47. Personalized Abstractive Summarization by Tri-agent Generation Pipeline

    Wen Xiao, Yujia Xie, G. Carenini, Pengcheng He

    2023Findings8 citations Open access

    Semantic Scholar

    Save

48. CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

    Chen Liang, Pengcheng He, Ye-Long Shen, Wei-Zhu Chen +1 more

    2022Annual Meeting of the Association for Computational Linguistics6 citations Open access

    Semantic Scholar

    Save

49. Summarization with Precise Length Control

    Lesly Miculicich, Yujia Xie, Song Wang, Pengcheng He

    2023arXiv.org4 citations Open access

    Semantic Scholar

    Save

50. A Prototype-Oriented Clustering for Domain Shift with Source Privacy

    Korawat Tanwisuth, Shujian Zhang, Pengcheng He, Ming-Yuan Zhou

    2023arXiv.org3 citations Open access

    Semantic Scholar

    Save