Comment by sho_hn

I've been having some similar thoughts lately about some application spaces being more harmful than others.

Coding, for example, seems somewhat benign, because code has to fulfil clear metrics: Either it works or it doesn't. Either it performs or it doesn't. As long as you are able to provide those metrics, you can to some extent treat it as a black box without losing much from a system view (lifecycle, long-term maintenance, keeping things working, broader qualified efficiency like re-use are of course other stories).

But on the other hand, delegating decision-making and thinking to models feels plain harmful to me. I have so many stories around me from office settings these days: "We have to pre-pone meeting XYZ, and we don't have enough time to prepare, so I made this AI analysis <power point deck>". This was then mis-prompted to fit a foregone conclusion, no one has time to do it or capacity to refute a 2000 word slop deck (except by slopping back), and crazy stuff becomes plan of record. This is really going to be a problem for organizations ...