Comment by mikgp

I feel like there’s a weird conflicts in how folks mental model of LLM development.

It’s not the model I don’t trust, it’s myself. The model is wrong _all the time_ because - it’s easy to verify the code - it’s hard to verify that I knew what I was talking about when I prompted it.

So the idea that you can broadly speaking take the human out of the loop. I think suggests to me a level of consistency in the contextual environment that would probably never exist.

At some point it’s politics. The model can come up with a better answer than my boss, and then my boss can just ignore it. Taking the human out of the loop broadly speaking implies that we all agree on what we’re trying to optimize.