Pre-Computable Multi-Layer Neural Network Language Models
Jacob Devlina b, Chris Quirka b, Arul Menezesa b
- aMicrosoft Research Redmond , WA , USA
- bMicrosoft Research Redmond, WA, USA
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
In the last several years, neural network models have significantly improved accuracy in a number of NLP tasks.However, one serious drawback that has impeded their adoption in production systems is the slow runtime speed of neural network models compared to alternate models, such as maximum entropy classifiers.In Devlin et al. (2014), the authors presented a simple technique for speeding up feed-forward embedding-based neural network models, where the dot product between each word embedding and part of the first hidden layer are pre-computed offline.However, this technique cannot be used for hidden layers beyond the first.In this paper, we explore a neural network architecture where the embedding layer feeds into multiple hidden layers that are placed "next to" one another so that each can be pre-computed independently.On a large scale language modeling task, this architecture achieves a 10x speedup at runtime and a significant reduction in perplexity when compared to a standard multilayer network.
TL;DRA neural network architecture where the embedding layer feeds into multiple hidden layers that are placed “next to” one another so that each can be pre-computed independently, which achieves a 10x speedup at runtime and a significant reduction in perplexity when compared to a standard multilayer network. (Semantic Scholar)