Comment by twoodfin
1 month ago
ChatGPT does not “write well” unless your standard is some set of statistical distributions for vocabulary, sentence length, phrase structure, …
Writing well is about communicating ideas effectively to other humans. To be fair, throughout linguistic history it was easier to appeal to an audience’s innate sense of authority by “sounding smart”. Actually being smart in using the written word to hone the sharpness of a penetrating idea is not particularly evident in LLM’s to date.
Good writers use words to make a point. LLMs use words to make a salad.
Depends what you ask the LLM to do!
If you're using it to write in programming language, you often actually get something that runs (provided your specifications are good - or your instructions for writing the specifications are specific enough!) .
If you're asking for natural language output ... yeah... you need to watch it like a hawk by hand - sure. It'd be nice if there was some way to test-suite natural language writing.
The last time I asked it to write something in a programming language, it put together a class that seemed reasonable at first blush, but after review found it did not do what it was supposed to do.
The tests were even worse. They exercised the code, tossed the result, then essentially asserted that true was equal to true.
When I told it what was wrong and how to fix it, it instead introduced some superfluous public properties and a few new defects without correcting the original mistake.
The only code I would trust today's agents with is so simple I don't want or need an agent to write it.
2 replies →
But they will make the salad delicious, marvelous and intricate. It's not just a salad - it's a new way to talk like marketing copy (/s)