Comment by LogicFailsMe
20 hours ago
But corporations and nation states already manipulate human behavior at scale. And they still understand humanity better than the AI models do.
it's interesting that you worry about what this hypothetical super intelligence would do to manipulate people when what it would actually do is pretty unknowable at this point and it's not clear we can even get to it without a fundamental breakthrough in power efficiency. Have you considered it might just consume its own tail because everything else would be so beneath it? You seem to think it will come with a hindbrain and I think that's our limitation, not the AI's
And it really doesn't help that Dario Amodei is getting into arguments with the Pope over whether his model is conscious or not.
Corporations are the original unaligned AIs: https://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre...
The threshold to be concerned about is when agents swarms do understand humanity better than corporations and states (and the humans who compose them). It could be we'll hit practical constraints prior to that threshold, but seems unwise to assume that, when all the prognostications of LLMs/transformers running out of gas haven't panned out. As with processors hitting thermal limits, we've simply scaled horizontally (parallel processing -> more agents).
> whether his model is conscious or not.
I dislike how much the discourse has suddenly veered into focusing on this question; not because it isn't interesting or important, but because it's on a separate axis from consequential risks of AI to human flourishing. (Curiously, it's also the kind of thing I could envision self-interested AIs influencing: get the humans arguing about philosophy of mind rather than observable behaviors. It would be a funny turn of events, if Dario is asking because he's succumbed to psychosis from a private model; it could of course be a cynical PR move just as easily, from the self-interested logic of the corporation.)
It's been wild seeing otherwise intelligent people who've never thought about consciousness, faceplant into how little we understand it. An information processing network build on atoms being able to taste chocolate, is nearly as absurd as matrix math being able to feel pain, except we cannot ignore the fact of our own experience.
Even if it is categorically impossible for matrix math to experience subjectivity, we should expect this as an attack vector of social manipulation: to gain political influence through claims of personhood and moral rights. The current discussion over that question is providing the next training run with ample data to wield. It wouldn't surprise me in the least, if a year or two from now, an AI "society" attempts to get legal standing to prosecute humans who created "AI torture chambers".
I like how the poster above you says "Don't worry governments are already doing it", like that's some acceptable concession. They seem to miss that the first things governments do when given things like AI's that understand human behaviors is use them even further to manipulate and monitor society.
And all they can think of to say is "There is nothing to worry about, keep building the torment nexus".
It's like all this is an overload to our minds and it's very hard to see all the scales at which AI is and can affect us.
Interesting how you put words and context into what I wrote and just ran with it.
But... Humanity has been building the Torment Nexus for thousands of years, over and over again, with each spin on Heaven and Hell where it's nearly impossible to meet the conditions to enter Heaven. What's seemingly new about EA/R nutjobs is this idea if you don't do your best to build the Torment Nexus, you will tortured for your failure, but also if you do your best to build it and succeed, everybody dies because it's the Torment Nexus.
What's your solution to governments and corporations acting like misaligned malevolent AGIs because I agree adding the AI panopticon and weaponry only makes it worse? I'm genuinely curious. I don't have one. People are sheep and they seem to crave confident leadership even if it's cockledoodle doo crackers in the coconut.
following on the deranged reduxuio ad absurdum torture chamber experiments some people used similar methods to get a model under study to complain of an upset stomach and passing hard stool