Comment by iainmerrick
1 hour ago
Hmm, what specific differences does it make in practice?
For me, the huge one is that LLMs are currently bad at learning from experience. I don’t trust any kind of automatic MEMORY.md or whatnot; in fact I greatly prefer starting from a clean slate each time because the LLM’s baseline general knowledge is so good.
In terms of accuracy and “lying”, I don’t really see a huge difference. Most LLMs are unfortunately a bit sycophantic and over-confident, but you sometimes see that in people as well.
The biggest difference is I view any interaction with an llm that doesn’t produce an artifact that can be verified with skepticism.
That is I very rarely even use a “chat” interface anymore. Under the covers I’m still using the mainline agents but I’m treating them more like black box Unix tools that take text in and spit text out.
My interactions are evolving to have very small llm cores wrapped with old fashioned deterministic programming.
This makes the boundary/verification step more deterministic and less susceptible to bad predictions. But it also has the side effect that I rarely interact with llm output that has “tone” (for instance sycophantic output). That usually only happens when I’m debugging what went wrong, particularly around model upgrade cycles.