Comment by xg15

I think the author oversimplifies the inference loop a bit, as many opinion pieces like this do.

If you call an LLM with "What is the meaning if life?", it will return the most relevant token, which might be "Great".

If you call it with "What is the meaning if life? Great", you might get back "question".

... and so on until you arrive at "Great question! According to Western philosophy" ... etc etc.

The question is how the LLM determines that "relevancy" information.

The problem I see is that there are a lot of different algorithms which operate that way and only differ in how they calculate the relevancy scores. In particular, there are Markov chains that use a very simple formula. LLMs also use a formula, but it's an inscrutably complex one.

I feel the public discussion either treats LLMs as machine gods or as literal Markov chains, and both is misleading. The interesting question, how that giant formula of feedforward neural network inference can deliver those results isn't really touched.

But I think the author's intuition is right in the sense that (a) LLMs are not living beings and they don't "exist" outside of evaluating that formula - and (b) the results are still restricted by the training data and certainly aren't any sorts of "higher truths" that humans would be incapable of understanding.