Noam Shazeer

Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit +4 more
202578K citations Open access
OpenAlex
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel, Noam Shazeer, Roberts, Adam, Katherine Lee +5 more
2019arXiv (Cornell University)11K citations Open access
OpenAlex
PaLM: Scaling Language Modeling with Pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma +63 more
2022arXiv (Cornell University)2,555 citations Open access
OpenAlex
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer
2015arXiv (Cornell University)1,621 citations Open access
OpenAlex
Image Transformer
Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Łukasz Kaiser +3 more
2018arXiv (Cornell University)1,085 citations Open access
OpenAlex
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
William Fedus, Barret Zoph, Noam Shazeer
2021arXiv (Cornell University)987 citations Open access
OpenAlex
Exploring the Limits of Language Modeling
Rafał Józefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer +1 more
2016arXiv (Cornell University)849 citations Open access
OpenAlex
LaMDA: Language Models for Dialog Applications
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer +56 more
2022arXiv (Cornell University)778 citations Open access
OpenAlex
How Much Knowledge Can You Pack Into the Parameters of a Language Model?
Adam Roberts, Colin Raffel, Noam Shazeer
2020Conference on Empirical Methods in Natural Language Processing (EMNLP)637 citations Open access
OpenAlex
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis +3 more
2017arXiv (Cornell University)622 citations Open access
OpenAlex
Generating Wikipedia by Summarizing Long Sequences
Peter J. Liu, Saleh, Mohammad, Etienne Pot, Ben Goodrich +3 more
2018arXiv (Cornell University)578 citations Open access
OpenAlex
End-to-end text-dependent speaker verification
Georg Heigold, Ignacio López Moreno, Samy Bengio, Noam Shazeer
2016IEEE International Conference on Acoustics Speech and Signal Processing532 citations
OpenAlex
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
Noam Shazeer, Mitchell Stern
2018arXiv (Cornell University)455 citations Open access
OpenAlex
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen +5 more
2020arXiv (Cornell University)411 citations Open access
OpenAlex
Music Transformer: Generating Music with Long-Term Structure
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer +6 more
2019International Conference on Learning Representations330 citations
OpenAlex
GLU Variants Improve Transformer
Noam Shazeer
2020arXiv (Cornell University)302 citations Open access
OpenAlex
One Model To Learn Them All
Łukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani +3 more
2017arXiv (Cornell University)253 citations Open access
OpenAlex
Mesh-TensorFlow: Deep Learning for Supercomputers
Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran +8 more
2018arXiv (Cornell University)234 citations Open access
OpenAlex
Tensor2Tensor for Neural Machine Translation
Ashish Vaswani, Samy Bengio, Eugene Brevdo, Chollet, Francois +9 more
2018arXiv (Cornell University)205 citations Open access
OpenAlex
Fast Decoding in Sequence Models using Discrete Latent Variables
Łukasz Kaiser, Aurko Roy, Ashish Vaswani, Niki Parmar +3 more
2018arXiv (Cornell University)169 citations Open access
OpenAlex
Music Transformer
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer +6 more
2018arXiv (Cornell University)144 citations Open access
OpenAlex
Corpora Generation for Grammatical Error Correction
Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer +2 more
2019Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)130 citations Open access
OpenAlex
HydraNets: Specialized Dynamic Architectures for Efficient Inference
Noam Shazeer, Kayvon Fatahalian, William R. Mark, Ravi Teja Mullapudi
2018IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings124 citations
OpenAlex
Fast Transformer Decoding: One Write-Head is All You Need
Noam Shazeer
2019arXiv (Cornell University)101 citations Open access
OpenAlex
A probabilistic approach to solving crossword puzzles
Michael L. Littman, Greg A. Keim, Noam Shazeer
2002Artificial Intelligence79 citations Open access
OpenAlex
Do Transformer Modifications Transfer Across Implementations and Applications?
Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus +12 more
2021Conference on Empirical Methods in Natural Language Processing (EMNLP)78 citations Open access
OpenAlex
Talking-Heads Attention
Noam Shazeer, Zhenzhong Lan, Youlong Cheng, Nan Ding +1 more
2020arXiv (Cornell University)55 citations Open access
OpenAlex
An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer +4 more
2018arXiv (Cornell University)54 citations Open access
OpenAlex
Swivel: Improving Embeddings by Noticing What's Missing
Noam Shazeer, Ryan Doherty, Colin Evans, Chris Waterson
2016arXiv (Cornell University)53 citations Open access
OpenAlex
ST-MoE: Designing Stable and Transferable Sparse Expert Models
Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du +4 more
2022arXiv (Cornell University)51 citations Open access
OpenAlex
Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$
Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra +39 more
2022arXiv (Cornell University)46 citations Open access
OpenAlex
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen +5 more
2021International Conference on Learning Representations43 citations
OpenAlex
GSPMD: General and Scalable Parallelization for ML Computation Graphs
Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Blake A. Hechtman +12 more
2021arXiv (Cornell University)40 citations Open access
OpenAlex
Primer: Searching for Efficient Transformers for Language Modeling
David R. So, Wojciech Mańke, Hanxiao Liu, Dai, Zihang +2 more
2021arXiv (Cornell University)39 citations Open access
OpenAlex
Searching for Efficient Transformers for Language Modeling
David R. So, Wojciech Mańke, Hanxiao Liu, Zihang Dai +2 more
2021neural information processing systems37 citations
OpenAlex
Blockwise Parallel Decoding for Deep Autoregressive Models
Mitchell Stern, Noam Shazeer, Jakob Uszkoreit
2018arXiv (Cornell University)36 citations Open access
OpenAlex
Proverb: the probabilistic cruciverbalist
Greg A. Keim, Noam Shazeer, Michael L. Littman, Sushant Agarwal +6 more
199936 citations
OpenAlex
High Resolution Medical Image Analysis with Spatial Partitioning
Le Hou, Youlong Cheng, Noam Shazeer, Niki Parmar +5 more
2019arXiv (Cornell University)27 citations Open access
OpenAlex
Solving crossword puzzles as probabilistic constraint satisfaction
Noam Shazeer, Michael L. Littman, Greg A. Keim
199927 citations
OpenAlex
Sparse non-negative matrix language modeling for skip-grams
Noam Shazeer, Joris Pelemans, Ciprian I. Chelba
2015Interspeech (USB)18 citations Open access
OpenAlex
Weakly Supervised Grammatical Error Correction using Iterative Decoding
Jared Lichtarge, Christopher Alberti, Shankar Kumar, Noam Shazeer +1 more
2018arXiv (Cornell University)17 citations Open access
OpenAlex
Faster Transformer Decoding: N-gram Masked Self-Attention
Ciprian I. Chelba, Mia Xu Chen, Ankur Bapna, Noam Shazeer
2020arXiv (Cornell University)12 citations Open access
OpenAlex
Skip-gram Language Modeling Using Sparse Non-negative Matrix Probability Estimation
Noam Shazeer, Joris Pelemans, Ciprian I. Chelba
2014arXiv (Cornell University)11 citations Open access
OpenAlex
NN-Grams: Unifying Neural Network and n-Gram Language Models for Speech Recognition
Babak Damavandi, Shankar Kumar, Noam Shazeer, Antoine Bruguier
2016Interspeech (USB)9 citations Open access
OpenAlex
Solving crosswords with PROVERB
Michael L. Littman, Greg A. Keim, Noam Shazeer
1999National Conference on Artificial Intelligence6 citations
OpenAlex
Sparse Non-negative Matrix Language Modeling
Joris Pelemans, Noam Shazeer, Ciprian I. Chelba
2016Transactions of the Association for Computational Linguistics5 citations Open access
OpenAlex
End-to-End Text-Dependent Speaker Verification
Georg Heigold, Ignacio López Moreno, Samy Bengio, Noam Shazeer
2015arXiv (Cornell University)4 citations Open access
OpenAlex
Pruning sparse non-negative matrix n-gram language models
Joris Pelemans, Noam Shazeer, Ciprian I. Chelba
2015Interspeech (USB)4 citations Open access
OpenAlex
Variational Program Inference
Georges R. Harik, Noam Shazeer
2010arXiv (Cornell University)3 citations Open access
OpenAlex
Do Transformer Modifications Transfer Across Implementations and Applications?
Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus +12 more
2021arXiv (Cornell University)2 citations Open access
OpenAlex

Most-cited papers