Comment by DennisP

Transformers are general-purpose learning machines. Guessing the next token is what we initially train them to do, but then we add other training on things like giving answers that humans like, or doing math that's judged correct by automated systems. These days they're getting quite good at advanced mathematics.