Comment by godelski

  > and having a multi-domain expert on call to lean on.

I don't feel like this is an accurate description. My experience is that LLMs have a very large knowledge base but that getting them to go in depth is much more difficult.

But we run into the same problem... how do you evaluate that which you are not qualified to evaluate? It is a grave mistake to conflate "domain expert" with "appears to know more than me". Doesn't matter if it is people or machine, it is a mistake. It's how a lot of conartists work, and we've all seen people who are in high positions and we're all left wondering how in the world they got there.

  > The real cost limiting creative and engineering efforts isn't the one of making a bad choice, but that of undoing it.

Weird reasoning... because I agree and this is the exact reason I find LLMs painful to work with. They dump code at you rather than tightening it up, making it clear and elegant. Code is just harder to rebase or simplify when there are more lines. Writing lines has never been and never will be the bottleneck because the old advice still holds true that if you're doing things over and over again, you're doing it wrong. One of the key things that makes programming so amazing is that you can abstract out repetitive tasks, even when there is variation. Repetition and replication only make code harder to debug and harder to "undo bad choices".

Also, in my experience it is even difficult to get LLMs to simplify, even when explicitly instructing them to and pointing them to specific functions and even giving strong hints of what exactly needs to be done. They promptly tell me how smart I am and then fail to do any of that actual abstraction. Code isn't useful when you have the same function written 30 different places and 20 different files. That's way harder to back out of decisions. They're good at giving a rough sketch but it still feels reckless to me to let them actually write into the codebase where they are creating this tech debt.