Comment by atleastoptimal

A claim that LLM's can in a theoretical sense be 100% accurate all the time is not the same as the claim that scaling models with more compute/params will reduce hallucination. The former is a far stronger claim and I agree with the paper in that it probably isn't the case, but we don't rely on general reasoners (a.k.a. humans) to be 100% accurate all the time either.

> No, it can hold more floating point numbers.

Fallacy of composition. Just because an LLM is made up of floating point numbers doesn't mean its capabilities are limited to that of bare floating point numbers, in the same way that the individual faculties of a neuron don't preclude the human brain from emergent properties born from the synthesis of its synapses.