Comment by Kiro
20 hours ago
I trust a modern model implementing a standard feature like this much more than 99% of the people I've worked with.
People bash LLMs for overengineering but for this it's what you want. Taking extreme edge-cases into account that a human would never bother with and obsessing over security.
That’s not what you want at all.
I mostly vibe coded a queuing system to replace something we’re using at work (last week. Spent about $1500). Then I meticulously went through the code.
It was much harder to review because it was ultra defensive and included guards for tons of edge cases that weren’t possible.
Unnecessary abstractions for possible extension later. Useless indirection. Probably 3x as much code as there would have been if I’d written it by hand.
I didn’t one shot this. I kept a pretty tight leash on the AI. I had probably a dozen markdown files with of plans that I created over hours of back and forth with the AI and reviewed before each implementation round. I had automated reviews and quality gates etc…
What I found in review was that it was full of very subtle bugs that would have bitten hard in prod. Committing offsets asynchronously that would lead to dropped messages. Clock drift bugs that would lead to dropped messages or write amplification storms. Lack of back pressure in some stages of the pipeline that would cause notes to get silently OOM killed. Weird over-insistence on never crashing in most places that would mask systemic errors.
If I’d just shipped it without review, it would have mostly worked. But at the scale it’s going to be used (tens of thousands of messages per second) it would have caused production issues for months while we tracked down each of these issues.
> included guards for tons of edge cases that weren’t possible
It's not possible until it is. This is the justification lazy developers like we all are have been using leading to bugs down the road. This glorification of hand-made code is strange, like we weren't writing dirty code full of shortcuts and hacks all the time.
The number of possible edge cases if you include all possible future changes to code, input, or environment are infinite.
Overly defensive code is harder to read and change for both humans and LLMs.
And many times it makes debugging harder by moving or suppressing failures.
The issue with LLM guarding isn't that it's "excessive" in outputting edge case handling, it's that the result often ends up just suppressing an error that actually indicates there is a bug or that should be handled elsewhere in a different way.
While I've definitely experienced it I don't think this is as much of a problem anymore. It's very easy to add an instruction to projects where you want every error to lead to a top-level throw rather than be handled. I also find that when it does try to mitigate it does so gracefully with a path you would actually make if you had infinite time, but your instinct tells you it's overkill.