Comment by cyanydeez

I don't see how anyone could make a successful product build on cloud LLMs, even if you get a perfect workflow, you'll either be gouged with price rises, or lose out to model changes and context/prompt divergence. All this "prompt" nonsense is simply trying to play to the LLM audience, and no amount of imprecise prompt will negate the fundamental instability.

So yeah, you have to use a localLLM if you think there's a viable product to be had. Anyone whose been programming knows that once you get to the mile mark of a complete & finished project, it can be mothballed for decades generating utility and requiring limited maintenance. All that goes out the window if you require a cloud provider to remain stable for a decade.