Comment by bluesign
5 hours ago
The thing is why that this feels so good future is; it is a system with no constraints. A bit like Star Trek universe in Roddenberry's imagination. This kind of utopia can only be achieved with all honest actors, but in reality systems are usually designed around bad actors.
Even with all morally good actors locally, there is no guarantees for external forces. Thinking it hypothetically, even with global coordination ( all good actors ) there is not a proven path that would lead us to better place from any starting point from past.
It's probably more predictive to model actors as being neither good nor bad but constrained by various collective dilemmas, such as prisoners dilemma, the security spiral, tragedy of the commons, race dynamics, collective action or first mover problems, information asymmetries, the commitment problem, among others. Those are the hardest problems to solve because they're pathologies that result from the global, largely amoral structure rather than consequences of the individual exercise of morality.
In the AI case, each firm is in an arms race, and nobody can slow down without effectively collapsing due to positive gross margins only being viable with a frontier model that attracts marginal demand. An appeal to morality might have an impact but more effective action would be to address the structure that the AI companies are situated in that causes this dynamic in the first place. In practice, thats going to be a global agreement to slow down, and global regulations.
Yeah but this is a system problem; if we had this utopic system from the beginning we would not even have AI probably.
This sort of rationalization of evil is a core of technocratic support for Trumpism, I find, and has parallels to the evangelical prosperity gospel. Choice tenets:
It’s always cloaked in a veil of realism, but it’s just the classic 14-year-old-boy-just-got-introduced-to-the-prisoners-dilemma situation. There’s nothing philosophically interesting about it.
Ironically, these are often the same people denouncing multiculturalism, yet the culture they strive for is completely morally bankrupt.
And it's funny because the "realism" has been proven wrong over and over and over again for millennia. People do all sorts of selfless and generous things all the time! The entire premise is trivially disprovable by just going and asking a neighbor for some help with something.
That's not to say we should be naive about greed or malice existing or being powerful motivators (especially the former), but it is obviously not true that they're the only forces at play and therefore you are "just doing the logical thing" by succumbing to them. It's just the more destructive version of the same naiveté.
Seems you and I have together struck a nerve. Maybe our sentiments would have been better received in an alternate subthread, but it’s all I could think about while reading the parent / cousin comments.
I haven’t read the full Magnifica Humanitas yet, but I would be pleasantly surprised if he touched on not just dehumanization of the other, but dehumanization of the self. Expanding on your thought, succumbing to those forces under the guise of just doing the logical thing is in a way self-dehumanization - to believe you are only capable of the “logical” thing instead of the moral thing.
This was not my point at all. Maybe I could explained better, but main criticism I have is: you can bundle together objectives ( which are inherently good ) and create an utopia. But those cannot always be achievable.
Everything in life in trade-offs. Simple example is speed/quality/cost. I can tell easily:
- services should be cheap - services should be fast - services should be high quality
Now I created an utopia. Obviously this is amazing to listener. They agree. But is it achievable?
It is not saying greed is good or might makes right. But system means you need to construct from this ideals best outcome ( which comes at some trade offs)
Yeah there were probably more appropriate subthreads to have responded to. My point wasn’t quite neatly directed against yours.
To get from here to Roddenberry's communism, according to Roddenberry's lore, we passed through the Eugenics Wars, the Second Civil War, and then fifty years of World War Three and the 'post-atomic horror' before coming to our senses.