Comment by adamddev1
1 month ago
People keep using these analogies but I think these are fundamentally different things.
1. hand arithmetic -> using a calculator
2. assembly -> using a high level language
3. writing code -> making an LLM write code
Number 3 does not belong. Number 3 is a fundamentally different leap because it's not based on deterministic logic. You can't depend on an LLM like you can depend on a calculator or a compiler. LLMs are totally different.
There are definitely parallels though. eg you could swap out your compiler for a different one that produces slightly different assembly. Similarly a LLM may implement things differently…but if it works do we care? Probably no more than when you buy software you don’t care precisely what compiler optimisation were used. The precise deterministicness isn’t a key feature
With the llm, it might work or it might not. If it doesn't work, then you have to keep iterating and hand holding it to make it work. Sometimes that process is less optimal than writing the code manually. With a calculator, you can be sure that the first attempt will work. An idiot with a calculator can still produce correct results. An idiot with an llm often cannot outside trivial solutions.
> but if it works so we care?
It often doesn't work. That's the point. A calculator works 100% of the time. A LLM might work 95% of the time, or 80%, or 40%, or 99% depending on what you're doing. This is difference and a key feature.
I see. I’d call that fragility/reliability rather than deterministic but semantics I suppose.
To me that isn’t a show stopper. Much of the real world works like that. We put very unreliable humans behind the wheel of 2 ton cars. So in a way this is perhaps just programmers aligning with the messy real world?
Perhaps a bit like architects can only model things so far eventually you need to build the thing and deal with the surprises and imperfection of dirt