Comment by roenxi
9 hours ago
If we had a theoretical technique to identify the true and objective reality we'd use it in the courts and laboritories. There is no such technique, but what we do have is 2 techniques that seem work:
1) Has a certain standard of evidence been met?
2) Are the related arguments free of logical inconsistencies?
We can train the LLMs to do 2, and maybe even 1 to some extent (exactly what quality of evidence a computer can practically gather is limited). But that isn't going to get rid of hallucinations, for the same reason courts are hit-and-miss or the conclusions of studies often aren't very reliable. These techniques help, but sometimes they still get people to say things that, on close inspection, turn out to be nonsense. And those best-effort approaches are too much to expect for most questions an LLM will be handed which are informal, low stakes and don't need strong supporting evidence or logical rigour.
I think it is underestimated how many LLM-style hallucinations people themselves have. It just isn't obvious because most humans have a strategy of only repeating what the herd says after it has been socially vetted, which makes their individual eccentricities less obvious.
TLDR; I don't think it looks like an easy problem for RLVR, it looks technically unsolvable. Even making progress requires a philosophical breakthrough on the nature of truth so that the objective function can be established.
Well, I'd argue that this depends on the field you're investigating. Sometimes you have a way to identify objective reality and sometimes you don't. In mathematics the majority of the field is verifiable in this way. Coding a bit less as it's intersubjective, as and the ideal methodology is subject to taste.
But even in muddy fields of reality like medicine, there are objective facts to be found. When someone comes into an ER with chest pain, you often find a true, undeniable reason for why that is happening. If their lung has collapsed, a coronary artery is clogged or the aortic artery is dissecting, even if you don't find that out it tends to be clear in retrospect. The area of reality that becomes muddy is when use proxy signals to try to figure out who gets promoted to expensive/harmful examinations we can make final conclusions from, or the cases that don't fit cleanly into one bucket or the other. But very often, the gold standard truly is golden.
Of course, many realms of reality cannot be verified in this way. But I'd argue that there are quite a few that can.
> In mathematics the majority of the field is verifiable in this way.
Does mathematics count as not a hallucination though? Particularly in pure mathematics they take a certain pride coming up with wild concepts as unrooted as possible in anything relevant to human existence. The name of the game is purely about maintaining internal logical consistency - which is something an AI can do while hallucinating.
AI hallucinations in maths might be logically consistent or not be. But in that particular case it starts to get a bit iffy what we call it when someone imagines something that doesn't exist. This gets back to the thing where we can train AIs to be logically consistent, but we can't force that consistency to be grounded in any particular universe. Ie, it'll hallucinate but in a very well rationalised way - coincidentally mimicking how a number of mathematicians seem to approach life.
This is the central issue; there is a very real trade-off between facts and verifiablity. Mathematics is perfectly verifiable because it is fact free. We don't have a reliable general system to verify facts. We do have reliable systems for checking arguments (logic).
Mmmm, not sure I agree with this, although this is a topic where we would have to do a lot of groundwork to formulate our positions precisely in order to ensure we're actually discussing the same thing. My counterargument is that verified mathematics does exist. A lot of mathematical models of physics predicted the existence of stuff that experiments later verified, the higgs boson, antimatter and gravitational waves comes to mind. Terrence Tao did in fact make MR machines go faster simply by finding better maths, and the tumors those machines see can be cut out and touched.
Yes, there are mathematical concepts that seem to exist purely in the realm of mathematics, but maths often touches reality in a consistent way that reflect experimental results. This seems to imply that there is more to mathematics than just internal consistency. And the parts that do not correspond to any observation right now, might just reach out and touch reality in the future. It is possible to create logically consistent systems that have nothing to do with reality, but this is not the mathematics that most mathematicians are thinking about.
Observation is the final arbiter of fact. Maybe we don't have a general system to verify ALL facts, but many facts are 100% verifiable, although not most of them. "Beyond reasonable doubt" is of course the highest level of fact as far as the scientific method is concerned, but some facts are so far beyond reasonable doubt that you might as well just call them true. In the average living human body, there is a particular clump of tissues that consistently corresponds a concept most experts would describe as a "heart", and it does in fact pump blood. True fact.