Comment by lanyard-textile

I think a better way to view these numbers is that they are "confidence" levels, not probabilities in the sense you are interpreting.

You should never trust the machine. ;) But you can have a level of confidence with it, and you can know where it's strong and weak.

This is a language association machine, not a math and probability machine. It is specifically weak with any real math.

As simple as a coin flip is -- as simple as a child may be able to understand it -- the machine is specifically not designed to produce an accurate mathematical probability.

That's just not how LLMs work. It's not how they're designed. Nor these decision models which use identical principals.

All that aside... agreed regardless. Irreversible choices should never be finalized by such a volatile technology, regardless of how well we benchmark it. If there's even a chance it could be wrong, that is unacceptable.