Comment by fragmede
1 month ago
Is your kitchen contractor an unthinking robot with no opinions or thoughts of their own that has never used a kitchen? Obviously if you want a specific cabinet to go in a specific place in the room, you're going to have to give the kitchen contractor specifics. But assuming your kitchen contractor isn't an utter moron, they can come up with something reasonable if they know it's supposed to be the kitchen. A sink, a stove, dishwasher, refrigerator. Plumbing and power for the above. Countertops, drawers, cabinets. If you're a control freak (which is your perogative, it's your kitchen after all), that's not going to work for you. Same too for generated code. If you absolutely must touch every line of code, code generation isn't going to suit you. If you just want a login screen with parameters you define, there are so many login pages the AI can crib from that nondeterminism isn't even a problem.
At least in case of the kitchen contractor, you can trust all the electrical equipment, plumbing etc. is going to be connected in such a way that disasters won't happen. And if it is not, at least you can sue the contractor.
The problem with LLMs is that it is not only the "irrelevant details" that are hallucinated. It is also "very relevant details" which either make the whole system inconsistent or full of security vulnerabilities.
The login page example was actually perfect for illustrating this. Meshing polygons? Centering a div? Go ahead and turn the LLM loose. If you miss any bugs you can just fix them when they get reported.
But if it's security critical? You'd better be touching every single line of code and you'd better fully understand what each one does, what could go wrong in the wild, how the approach taken compares to best practices, and how an attacker might go about trying to exploit what you've authored. Anything less is negligence on your part.
You kitchen contractor will never cook in your kitchen. If you leave the decisions to them, you'll get something that's quick and easy to build, but it for sure won't have all the details that make a great kitchen. It will be average.
Which seems like an apt analogy for software. I see people all the time who build systems and they don't care about the details. The results are always mediocre.
> The results are always mediocre.
I think this is a major point people do not mention enough during these debates on "AI vs Developers": The business/stakeholder side is completely fine with average and mediocre solutions as long as those solutions are delivered quickly and priced competitively. They will gladly use a vibecoded solution if the solution kinda sorta mostly works. They don't care about security, performance or completeness... such things are to be handled when/if they reach the user/customer in significant numbers. So while we (the devs) are thinking back to all the instances we used gpt/grok/claude/.. and not seeing how the business could possibly arrive to our solutions just with AI and wihout us in the loop... the business doesn't know any of the details nor does it care. When it comes to anything IT related, your typical business doesn't know what it doesn't know, which makes it easy to fire employees/contractors for redundancy first (because we have AI now) and ask questions later (uhh... because we have AI now).
Especially if you get quotes and bid it out, you'll self select for people who care less
Most software, especially web software, is mediocre. LLMs are good at building web software.
Do all details matter equally? Maybe only 3/10 of them you really care about and you can leave the rest up to the contractor
That still requires you to evaluate all the details in order to figure out which you care about. And if you haven't built a kitchen before you, won't know what the details even are ahead of time. Which means you need to be involved in the process, constantly evaluating whether what is currently happening and if you need to care about it.
Which three? To which people? Congrats, you've discovered integration hell
Maybe they have a kitchen without dishwasher. So unless asked they won't include one. Or even make it possible to include one. Seems like a real possibility. Maybe eventually after building many kitchens they learn they should ask about that one.