Comment by mrguyorama

The physical responses of the world around them after you have yanked the concept outside of the human brain

We have to blind medical professionals during science because even thoroughly trained and experienced professionals are still more likely to form conclusions and opinions based on understood human biases than reality.

You can take a gambling addict and teach them as much statistics and probability as you want, and even if they demonstrably learned it, they will still go back to the slots and believe a hit is "due" because the link between reality and the brain's construction of its internal models is extremely limited, and those models only inform the brains processes, not necessarily constrain it.

I will never understand however how some people think that an LLM can pull a signal out of it's training material that doesn't actually exist in its training material.

It's like training an LLM on monopoly games and expecting it to be good at chess. What?