Comment by mrtesthah
19 hours ago
Probably good to establish canonical spec documents up front that it can use and maintain as an independent reference?
19 hours ago
Probably good to establish canonical spec documents up front that it can use and maintain as an independent reference?
That doesn't solve the problem of it hallucinating more design details and putting them in comments. The solution to too much authoritative documentation isn't more authoritative documentation.
Comments exist for humans to understand the why of the code, including perhaps external dependencies/assumptions/requirements.
LLMs inserting code comments makes zero sense by definition.
That’s not true because LLMs clearly benefit from having multiple statements representing aspects of the same thing.
That’s why they’re always doing the “it’s not X, it’s Y” kind of patterns. It’s reinforcement.
LLMs benefit from knowing the why, the intent, and the reason/justification for code to exist, too.
Catching code desyncing from it is a valuable place to reconsider assumptions and maybe even invariants.
> LLMs inserting code comments makes zero sense by definition.
Not completely, because anyone who reviews the code in the future does not have access to your original prompt, so theoretically leaving a comment that explains the "why" portion of the prompt would be useful. LLMs rarely do that, though.
Doesn't need to be "up front", but you do need to curate the context.
AI can't read your mind, and it will read what you (or anyone else) wrote.
Humans can’t read minds either, but most humans aren’t so dumb as to forget what is a normative spec and what is just their own current, evolving model of the problem space. (Though some humans certainly are…)
It can't but it will try and getting it to stop is often infeasible.