Comment by simianwords

Knowing which word is likely to come after the other is trivially the concept of knowing truth for me.

Why not? We have optimised for truth and we are predicting the best words that ensure this optimal value.