Comment by verisimi

I think everyone seems confused about chat gpt and ai in general. The issue is that it doesn't share our values. It can't.

Humans assume that what is being expressed relates to the truth as a basic. This is not chatgpt's goal. It wants to create something that gives the appearance of truth. It's aim is to sell or convince you of something; actually delivering the goods is incidental in its aim to provide what seems right.

Put simply, it generates bullshit - any truthful output it generates is accidental - its only attempting to get your 'sign off'.

The danger is that we all start to live in bespoke worlds curated for us by ai that cater to our individual predilections. As it would be working with our existing biases, we could end up living in an echo chamber that perpetually supports our biases with ever less engagement with reality. We may find it ever harder to discern reality from provided narratives.