Decoding with Large-Scale Neural Language Models Improves Translation
Ashish Vaswania b, Yinggong Zhaoc d, Victoria Fossume, David Chiangb e
- aUniversity of Southern California Department of Computer Science
- bUniversity of Southern California, Los Angeles, United States
- cNanjing University , State Key Laboratory for Novel Software Technology
- dNanjing University, Nanjing, China
- eUniversity of Southern California Information Sciences Institute
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
We explore the application of neural language models to machine translation.We develop a new model that combines the neural probabilistic language model of Bengio et al., rectified linear units, and noise-contrastive estimation, and we incorporate it into a machine translation system both by reranking k-best lists and by direct integration into the decoder.Our large-scale, large-vocabulary experiments across four language pairs show that our neural language model improves translation quality by up to 1.1 Bleu.
TL;DRThis work develops a new model that combines the neural probabilistic language model of Bengio et al., rectified linear units, and noise-contrastive estimation, and incorporates it into a machine translation system both by reranking k-best lists and by direct integration into the decoder. (Semantic Scholar)