Comment by twobitshifter

Ngram is compressing several layers of multiplication to a lookup which negates the need to have the same model depth and reduces the model size that must be loaded.