Comment by sneurlax

And yet they're trained on the corpus of human writing. They may not act like humans but they do act like human writing.

"If you don't make profit, your business will be closed" is a pretty clear ultimatum for an agent tasked with creating a profitable business.