AWD-LSTM-MoS + dynamic evaluation (WT2, 2017)
Carnegie Mellon University (CMU)Language modeling
Developed by Carnegie Mellon University (CMU) in 2017, AWD-LSTM-MoS + dynamic evaluation (WT2, 2017) is a language modeling model with 35000000.0 parameters.
About AWD-LSTM-MoS + dynamic evaluation (WT2, 2017)
We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models (including the majority of neural language models) is limited by a Softmax bottleneck. Given that natural language is highly co
Details
- Provider
- Carnegie Mellon University (CMU)
- Task
- Language modeling
- Parameters
- 35000000.0
- Released
- 2017-11-10
- Open weights
- No