Comment by _ache_

You are missing the point. It's more about the connection made with the user who trust the AI. Think about all the elders that fall for the "kid abroad who need money". The connection is more important than the facts.

The problem is not (not only) that it told a 50 something guy someone is coming to kill him. The problem is the bond with the user.

Everyone is fallible; you just need to craft the right narrative. Obviously, been educated about what AI makes you less affected, ..., by AI only.