Comment by maaaaattttt
7 hours ago
What's happening in the math field at the moment represents on a smaller scale the issue we (probably) will face when AI becomes smarter than us in general. Do we slow the AI down so we can understand what it does and if it's correct (and align with our values for bonus points); do we, humans, adjust the way we work to the new speed or do we give in and let the AI advance while we're not completely sure of what it does and the correctness of it. The math field is in the position of showing us the way.
I think the bigger issue is that "we" won't be doing any deciding. A handful of oligarchs will. As the technology becomes better than us, I am afraid we will learn that we are (at least seen as) technology as well. And the owner of the better technology will decide what to do with the inferior technology. I don't see anyone trying to design the roads for horse carriages, so I don't understand why would anything change for humans' understanding.
That is, if things go in the current trajectory. I don't see any reason why anything would change though.