Comment by kjellsbells

Scarily true:

Users assume a system is always correct if it has been mostly correct before.

This isn't a new phenomenon, although AI certainly brings it to the fore.