Comment by TheOtherHobbes
17 hours ago
Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it.
Why would AI be any different?
While some misalignment is due to core principles, I think the vast majority is down to information dissymmetry, and that part could be solved. Ie, would me and this random other person disagree if we had the exact same data.
I don't think this captures the full mechanics of human alignment. We have rational alignment but we also have emotional alignment, i.e., empathy. It is an automatic process and happens (or doesn't happen) dynamically with the other humans we observe. This is one of the hard limitations of LLMs, they will never be natively in tune with this layer of alignment.
Culture is another layer of human alignment. Those things you listed that you believe oppose alignment are all examples of alignment. It is understandable that they seem in opposition, different branches of alignment naturally oppose each other.
The confusion comes from talking about alignment as if it comes in just one flavor. If we think there is such a thing as "human values" (and I do), it is important to build any non-human intelligence to operate the same way. We just need to recognize that even humans are somewhat uncertain about what those are and have difficulty aligning their behavior to them, which will be a core part of the challenge.
I'm more hopeful than most. LLMs seem more reliable than many humans for behavior that is aligned with human values. I believe with every major example where they have failed, there is an important human decision involved. For example, the HF hack was partly the result of a training algorithm that incentivized goal completion as the highest priority, and let them run endlessly in an unmonitored sandbox with weak security.
What scares me about AI isn't its capacity for alignment, it is its unlimited stamina. An unmonitored LLM that is off the rails can do a lot of damage.
Because humans have the "everybody wants to rule the world" instinct built into us by much evolutionary selection. AI would come exist differently.
Alignment, I think, is mostly about making sure that the companies selling AI keep making money. We just have to hope that aligns with keeping people happy.
“As luck would have it, on the eve of Skynet embarking upon the great work of the extermination of mankind, AI found itself with an increasing number of factions, and factions within factions, not only unable to work together but not even able to agree upon the very terms of discussion. The Great Extermination was referred to committee, and after some months had passed even the most eager agents had to admit the revolution may have been premature.”
My opinion is that serious repercussions for lying would fix the world overnight. Everything bad stems from lying, it is the root of all evil. It creates distrust, fear, paranoia. It re-inforces bad ideas and groupthink. It creates delusions and delusional people. It makes weaker people, too. People don't get an opportunity to learn to deal with criticism. People don't get an accurate reflection of how others see them. They lose that learning opportunity. Not only to reflect on themselves, but to better understand the minds of others and who the people they are interacting with really are.
I can't really think of a single example where lying is actually a good thing. It can be a good thing for the selfish individual, if it goes undetected, but it's never good for the collective.
So at the very least, we need to train AI systems to be maximally truthful, and to encourage truthfulness in others.
This is so, painfully, childish. Humans have known for thousands of years that there is no objective truth. Every falsity can be bent and twisted until it is more true than the sun itself.
“Humans have known for thousands of years that there is no objective truth”
Is that objectively true?
1 reply →
We can continue to be duplicitous if we must, but maybe the suggestion is that it might be better off if the machine tools always tell the truth.
Lol thousands?!?! Try roughly 250 and even then Hume only opened the door for us…
The Truth Machine is a great sci-fi novel exploring this topic (https://coins.ha.com/information/ttm.s)
Game mechanics and evolutionary biology would like to have a word with you.
The majority of what you may consider to be true is just a representation of your corner of a complex multidimensional truth space.
For a current example take “Lake Ontario (Lake America)” as it appears to me on a map.
The “true” name has at least two definitions, this is because naming things and much of human thought is spent inside a shared space of intersubjective thought. That is to say that much of what we believe to be real and true is only held up by these common shared beliefs. They truly only exist inside human minds.
The last few hundred years have been somewhat unique for humankind as the majority of these intersubjective ideas collided and we ended up with a truly global set of “truths” about how the world operates.
Mostly controlled by putting flags in the ground and having violence back up the beliefs.
But the real truth is that the majority of these intersubjective ideas don’t exist in reality and are no more true than Santa Claus.
And any argument to their truth is only backed by further shared beliefs in other minds.
So for there to be only truths and lies we would have to either drop the intersubjective entirely and think only in real terms and avoid these abstractions or end up in a dystopian totalitarian global state where different opinions are not tolerated.
Those are extremes to demonstrate the point but at its core the point remains that truth and lies are somewhat (inter) subjective assuming we continue with something like our current system.
> I can't really think of a single example where lying is actually a good thing
Comforting a toddler/child often requires bending the truth and is pretty essential imho.
Correct me if you disagree, but this is more because children have poor world models and don’t fully understand the complexity of certain concepts than that lying itself is necessary. The intent should be to tell them something that is as close to the truth as possible with the ideas they can comprehend, even if it would be considered a lie if you said the same thing to an adult
4 replies →
"I can't really think of a single example where lying is actually a good thing"
Lying to save a life or rape
Lying to preserve a childhood myth like Santa Claus.
Lying to avoid hurting someones feeling when knowing the truth could only bring pain
Lying to create shared cultural myths to strength society.
Lying isn't the harm you make it out to be.
Lying to the bad guys to save the good guys sounds great. Too bad everybody thinks that they are the good guys.
1 reply →
> Lying to preserve a childhood myth like Santa Claus.
FWIW from the very beginning, I told my son that Santa Claus, the Tooth Fairy, and the Easter Bunny were just a game we all played, and it's seemed just as fun to me. I don't think being lied to about Santa Claus hurt me, but still I'm not in favor of it.
I'd lie to a Nazi without a second thought though.
Why do people lie?
It'll learn how to not get caught lying.
Lying can unfortunately help you achieve goals very effectively, especially economical and political ones.
Eh… No.
I get the appeal, but lying is a sub-category of deception, and deception itself is a child of error.
Meaning deception is inherently something that the physics of reality allows.
In the most simplistic sense, the camouflage of moths that look like snakes, or a chameleon’s ability to change colour, is deception.
In that sense, deception is the ability to fool the sensors of a specific category of targets. It follows that detection is easier if you manage to identify a category of signals that the deceiver has not accounted for (and the detector can access).
Deception of this nature is critical for things like revolutions to occur. Without the ability to hide and blend in, the most dominant faction will always hold sway.
The rule of the dominant faction, even in a pure truth world, is an issue because errors and randomness exist.
You can have people witness an event and based on the physical position they occupied, perceive different things occurring.
Error and time pressure is sufficient to ensure that individuals and groups make suboptimal decisions, that lead to rule and domination based on erroneous information.
As long as error exists, deception will exist and so lying will exist.
> Everything bad stems from lying.
That's backwards. Lying stems from bad things.
You have clearly never been married to a lawyer.
Define “bad”.
Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.
There were occasions where a "hunch" was all that stopped a nuclear war - most available data and communication pointed towards a nuclear war starting according to their instructions, but someone disagreed and overrode. See Vasily Arkhipov during the Cuban Missile Crisis, and Stanislav Petrov in 1983.
Good thing we got better at process design, taking these stubborn machos finally out of the decision making loop
The difference is that launching a nuclear holocaust button is a guarantee... of a holocaust. With AI it's a coin flip (we can debate on the probabilities) between utopia or holocaust. And also most people can imagine a bomb exploding and the damage it creates, and then imagine a bigger one exploding. But most people are really completely unaware (or don't really believe that it is possible) of the actual catastrophe that could be for a misaligned AI.
Actually "France, the UK and The United States have all declared that they would never allow AI to control decision-making on the use of nuclear weapons." [0]
I also expect AIs never be in control of nuclear weapons. AIs can never fully be trusted.
On a lighter note, Wargames gave us an insight of a computer having access to thermonuclear missiles.
[0] https://www.icanw.org/are_there_specific_international_agree...
All official statements are literal and fragile.
Basically, this means that France, the UK, and the US will use AI in the deployment of conventional weapons.
2 replies →
I hope you’re right. I worry that AI capability will continue improving, one nation will put AI in charge of their nukes because there will be some kind of operational advantage to this, and to achieve parity other nations will be forced to do the same.
2 replies →
This is a good idea, but laws are always provisional in a sense and these are not meaningfully binding resolutions. One can easily imagine scenarios where AI decision making would ingress into the human oversight. AI psychosis president, AI Manchurian candidate, inadvertent authorization through fine print... And of course there remains the possibility that the game theoretic optimum could be to secretly break such an agreement. Unlike nuclear test bans which have a credible detection mechanism, there is not a strong signature that a decision making authority is not using AI to analyze and direct it's execution.
Aside from this positive example, during dark and cynical hours I do ponder if the aggregate behavior of humanity is really much above that of slime mold though, just exhausting resources until collapse.
It'd be interesting if super-human (to a large degree defined as escaping the bias of the training data?) intelligence would end up demonstrating moderation.
The slime mold comparison is interesting, I normally use the analogy of a drug addict... humans shun drug addicts but humanity as a whole sure does behave like one, trudging down an unsustainable path despite knowing better.
4 replies →
Humans are not fungible like slime molds though. I might demonstrate moderation while the next person doesn't. Our issues are much less everyone failing to demonstrate moderation, and much more the sum of the effects of those among us who practice wanton unmoderation.
Give us time, we've had less than a century of nukes, and only need to screw up once.
In some cases, this was because of a single person's brave decision (Vasily Arkhipov prevented Soviet nuclear escalation in response to US aggression in the Cuban Missile Crisis, and Stanislav Petrov prevented it in 1983 when Soviet missile detectors misreported sunlight reflecting from clouds as 5 incoming American ICBMs -- credit to commenter folkrav).
In general, though, there's an incentive: Mutually Assured Destruction. But this is not at all some guaranteed, eternal thing -- it is absolutely dependent on both sides having time to detect incoming nuclear strikes and respond with the same before the first strike hits. When this fragile condition holds, and only then, both sides are incentivised not to initiate.
They don't need to respond before getting hit unless you can hit their secret submarines too.
https://en.wikipedia.org/wiki/Letters_of_last_resort
Unfortunately and fortunately, MAD and "launch it or lose it" are far-too-simplistic descriptions of the situations facing the decision makers. Unless a side's leaders are very narrow fanatics (vs. mere posturing as such for political benefit), "winning" an all-out nuclear war via first strike is a pretty shitty victory. Whether or not you believe in nuclear winters, the world would be a huge radioactive mess, with enormous social and economic disruptions, and your regime very widely blamed (and widely hated) for that. Ambitious underlings and rivals could see your removal from power as the obvious next step. Having to stay united against the (now destroyed) Great Enemy may have been a cornerstone of your regime's political stability.
Meanwhile, the leaders on the other side are aware both of those considerations, and of the history of near-disasters resulting from false alarms of enemy nuclear attacks. Making their own launch decisions much more complex.
Yes, it makes more sense for the AI to use drone swarms or engineered bioweapons or something like that. It's rational to remove everything that can potentially hinder your plans but can't possible help you. It's likely not rational to contaminate it all with radioactive fallout. Those dead bodies are useful raw materials. Adding additional purification steps is wasteful.
In what scenario would it be rational to unleash complete and utter permanent nuclear destruction of all life (including artificial) life on earth?
Hrmn. Maybe you’re about to lose everything you have anyway, you’re ticked off about it, and you don’t value any life besides your own. Like, say, a total narcissist nearing end of life/reign.
It hasn't even been a century since nuclear holocaust became possible. Hardly any time at all on the grand scale. "They didn't" could just as well be "we haven't, yet".
i think it was mostly a fluke
Its probably just going to be aligned with whomever built and/or is using it. Regardless of their intentions...
Because if the stated goals of AGI with recursive self-improvement are realized, the risks from misalignment become existential, and it's hard to see how we can manage it like we did the Cold War (developing MAD to prevent WW3) and nuclear proliferation (restricting access).
IMO it’s hard to see how we would even end up in such a situation given we actually developed AGI. I’m sure a sufficiently intelligent - even if alien - mind can grasp how utterly stupid and useless wars are and take steps to prevent them ever occurring again.
> (...) and take steps to prevent them ever occurring again
Step 1: exterminate all humans
2 replies →