Comment by gwern

https://www.gwern.net/GPT-3#bpes for further reading. https://arxiv.org/abs/2212.10562#google is a particularly impressive recent demonstration of how serious the issue is.