Comment by astrange
15 days ago
AI writing style comes from post-training, which is entirely under the control of the labs. It's not copying it from any public sources.
15 days ago
AI writing style comes from post-training, which is entirely under the control of the labs. It's not copying it from any public sources.
I didn't say it's copying anything. It's learning styles - all of them - in pre-training. Post-training is just about giving it the right defaults.
Doesn't change my point anyway - even if post-training, labs naturally want the LLM to come off as smart (well, except for Grok). Post-training is where they "untrained" em-dashes and such, due to complaints of users.
I'd heard an interesting theory that in training on customer chats there's a lot of interactions like this:
Agent: _wall of text_
User: yeah sounds great, do it
Which has the predictable outcome of biasing towards wall of text responses.