Comment by adjfasn47573
4 hours ago
> A human must verify everything an LLM presents as fact. Everything.
I've thought about this for quite some time now.
No. A human doesn't need to verify everything. And the argument is really simple: stochastic.
Think of self-driving cars: We can show today - based on evidence and real data - that self-driving cars are safer than human drivers. That's a fact and the consequences are clear, more self-driving cars, less human-driven cars, less accidents, less hurt people, less dead people.
Are the cars 100% safe and NEVER make a mistake? No. But they don't need to. Nothing is ever 100% (in the real world).
Now back to AI for software creation. "Review is the bottleneck because EVERYTHING must be judged by a human." No. It doesn't. We just need to build AI review systems, that will do reviews better than (or at least as good as) humans. The human review quality bar is far below 100%. Far far far. If we can show (likely in the next 12-24 months I think) that AI review quality is consistently above the human review quality - again, based on evidence, based on real data - then that's it, then there's no good reason to have humans review the code.
Yes, there will be another layer in the system, another level of abstraction that will/must end at the human boundary.
> Now back to AI for software creation. "Review is the bottleneck because EVERYTHING must be judged by a human." No. It doesn't. We just need to build AI review systems, that will do reviews better than (or at least as good as) humans. The human review quality bar is far below 100%. Far far far. If we can show (likely in the next 12-24 months I think) that AI review quality is consistently above the human review quality - again, based on evidence, based on real data - then that's it, then there's no good reason to have humans review the code.
Let me know once the majority of software engineering organizations start only checking in markdown files and let code be generated non-deterministically from these specs in CI. If this is not happening now, there’s clearly a sufficiently high level of distrust in blind LLM output (both code itself and reviews).
And even then, are you suggesting humans shouldn’t at least review the markdown specs? Why not have LLMs review the specs then? Is there, perhaps, some fundamental quality to human review process that is desirable here?
This reduction of everything to stochasticity is silly. Or, to put it differently: Do you accept a value with some error appearing in your bank account on salary day?
We have plenty of systems where complete accuracy is the only acceptable thing. Computers are great for such things. Until we all get caught up in a way of delusion and start writing those systems as natural prose passed through an improperly understood stochastic machine.