Comment by Swizec
8 years ago
> I realized that there is no pleading or reasoning with The Machine.
Weapons of Math Destruction was one of the most eye opening books I read recently. It's all about how Machines when set up with self-reinforcing models can become a real big problem.
And how the fact nobody quite understands how The Machine makes decisions isn't helping either.
Examples of these abound, from the article to the Ethereum DAO "hack" and too often (just look a bit lower in the comment section) the response is "The Machine is working as it should." I feel like automation is great but don't forget that people are still people and someone should be able to lift the lid and fix things themselves.
My solution would be to not rely too much on these systems, but at the very least, at least include escape hatches and big red emergency buttons. Engineers do, why can't hackers do this?
> Engineers do, why can't hackers do this?
That can be difficult if security features rely on humans not being able to subvert the process, I think. But for other processes, this should be the default!