Comment by simpaticoder

I have a theory that most people at most levels of work deal with the same amount of mental complexity. The difference is that as you move up the unit of abstraction gets larger, so your decisions and knowledge cover more scale. E.g. the engineer thinks about functions; the business owner thinks about products; the investor thinks about companies; the president thinks about nations; etc.

But people at the higher abstraction levels have a problem because they often never had (or lose) the ability to zoom in anywhere. And even the ones that can don't have time to learn to zoom in everywhere so they have to learn how to trust others, aka recognizing shibboleths. AI is great at sounding trustworthy and making reasonable looking output. In so doing, however, it's undermining the utility of shibboleths for large scale thinkers! That is the powerful are now deluding themselves that they have access to infinite reliable experts, and have gained the ability to zoom in everywhere, for only the cost of a data center. In a sense programmer experts like us are lucky because we have objective verification as a feedback loop to temper the exuberance. They do not.

If this is true, the kinds of error modes we'll see will be novel and catastrophic to a large fraction of businesses. If the feedback loops for correction are damaged or destroyed, we'll see firms gleefully, optimistically and energetically committing obvious mistakes until they die.