Comment by cassianoleal
9 hours ago
What happens when your LLM of choice goes on an infinite loop failing to solve a problem?
What happens when your LLM provider goes down during an incident?
What happens when you have an incident on a distributed system so complex that no LLM can maintain a good enough understanding of the system as a whole in a single session to spot the problem?
What happens when the LLM providers stop offering loss leader subscriptions?
AFAIK everything I use has timeouts, retries, and some way of throwing up its hands and turning things back to me.
I use several providers interchangeably.
I stay away from overly complex distributed systems and use the simplest thing possible.
I plan to wait for some guys in China to train a model on traces that I can run locally, benefitting from their national “diffusion” strategy and lack of access to bleeding-edge chips.
I’m not worried.