Comment by brap

I learned that every step that can be solved reasonably without an LLM, should be solved without an LLM. Reliability, cost, performance, etc.

I try to transfer as much work as I can out of LLMs and into deterministic steps. This includes most of the “orchestration” layer which is usually deterministic by nature.

Sprinkle a little bit of AI in the right places and you’ll get something that appears genuinely intelligent. Rely too much on AI and it’s dumb as fuck.

Make their tasks very small and simple (ideally, one step), give them only the context and tools that they need and nothing else, and provide them with feedback when they inevitably mess up (ideally, deterministically), and hope for the best.