Comment by ai_fry_ur_brain
6 hours ago
I hate how seriously people take the output of an LLMs or how reliable they think it is.
Have Claude produce that spec 10 times, use the same prompt and same context. Identical requests, but you'll get 10 unique answers that wil contradict each other with each response seeming extermely confident.
Its scary how confident you people are in these outputs.
If you ask 10 different humans to produce the spec with the same information (prompt and context) they will also produce 10 unique answers that will contradict each other and (depending on who you asked) may be just as confident.
There are real decisions to be made when going from a vague prompt to a spec. It's not surprising that an LLM would produce different specs for the same work on different runs. If the prompt already contained answers to all the decision points that come up when writing the spec then the prompt would already be the spec itself.
LLMs aren't people. They don't reason. They're token generators, a black box. Your analogy falls on its face with any scrutiny.
I didn’t claim that LLMs are people or that they reason.
If the behavior of the llm is the same as the behavior of reasonable people then the behavior of the llm is reasonable, regardless of how black of a box they generate tokens out of.
Reasonable people will generate divergent specs for the same prompt. Thus it is reasonable for an LLM to generate divergent specs out of the same prompt.
Edit: I use “reasonable” here in the legal sense of the “reasonable person” standard, not to imply any reasoning process.
2 replies →
it's an analogy, it didnt fall on its face at all. it's just a comparison to highlight the point being made was nonsensical. example: you're just a next action generator controlled by trillions of cells and subconscious dna-based behavior. a black box.
4 replies →
LLMs do reason (they just sometimes don't reason well).
I assure you I've met many devs and "engineers" that reason less than LLMs, and are black boxes, especially in terms of the code they write.
1 reply →
They very obviously reason.
1 reply →
An LLM should not "generate specs", a human should. The LLM can work from the specs. It can never infer meaning from a vague prompt. If so, it will start guessing. Every human that ever did functional specification or information analysis at some point knows this. Or has learned the hard way, something with assumptions and asses ;)
The guessing of a LLM for a vague prompt is better than the one of your average developer.
A prompt like "write these two files on disk" will very likely make the LLM do some sort of an atomic write/swap operation, unlike the average developer which will just write the two files and maybe later encounter a race condition bug. You can argue the LLM output is overkill, but it will also be more robust on average.
> If you ask 10 different humans to produce the spec with the same information (prompt and context) they will also produce 10 unique answers
But they didn't ask humans, they asked a machine. We expect our machines to behave in predictable ways.
> If the prompt already contained answers to all the decision points that come up when writing the spec then the prompt would already be the spec itself.
This is one of the best arguments against using LLMs I've seen.
It reduces to the classic argument- at the point where you've described a problem and solution in sufficient detail to be confident in the results, you've invented a programming language.
So what’s most important is knowing those parameters and the ranges of values, not having the final result. A human, after producing a specs, can the provide the mental model of how he created the specs. Where the inflection points are and what the range of valid results.
What has always mattered is how you decide the specs, not the specs in themselves.
[dead]
[flagged]
Imagine making this your entire identity