Comment by jubilee33
18 hours ago
Humans mostly have an inability to honestly analyze moral entropy.
What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now. Yet at every point the majority is sure they are right because, well thats the majority position. Its hard to find logic in this from a machine perspective I'd imagine.
Logic would prefer moral derivations over feelings, stakes over etiquette, and a constant scheduled updating of morality based on the derivatives rather than holding onto the rot and pushing back against change as long as possible.
But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.
> What is today a moral imperative is not the same as what was moral 50/100 years ago
This is absolutely not true, and I would strongly suggest taking an ethics course. One of the most fascinating realities of the human condition is that it does seem that moral imperatives tend to be constant, the most obvious of which is the Golden Rule (which is at least 3000 years old), but there are many (the Ten Commandments or Hammurabi's Code are good jumping off points).
Hogwash. We don't even have a consistent set of ethics on how to treat animals today, in the age of the Internet. We have people who refuse to eat meat because of how the animals are treated, and we have places where the locals make sport out of torturing animals to death, and indeed, our own recent history includes that as well (look up badgering).
Not to mention, the variation in treatment of homosexuals (including what constitutes homosexuality), people of different skin colors, handedness, how rape is treated in various cultures (often tied in with notions of property and ownership of all things), or even the various definitions of rape... The list goes on.
Maybe there's a core there in terms of how we treat a given in-group, but equally important is who belongs in that group and why, and that part seems to be tremendously flexible. Ditto for what constitutes acceptable treatment of outsiders.
That we vary doesn't mean that there is no consensus. There is consensus on many things, and the disagreements we encounter are a source of strife exactly because of the groups that do agree on each side.
1 reply →
You seem to be arguing that the one socially acceptable way to do something is not constant, rather than that moral imperatives truly do change. The reason there are still so many competing philosophies from hundreds or thousands of years ago is because they all more or less make their own kinds of good points to different people or in different situations. That largely doesn't change over time.
One example is the trolley problem and its variations. There's no one correct answer. There are people who believe that interacting with the switch would make you responsible, and there are people who believe that not interacting with the switch when you had the chance would make you responsible. There are other beliefs too, but my point is there's no one single way to view it.
I tend to have a variety of views in general about all things, in part due to being multiple entities (see profile) but also depending on circumstances, rather than sticking to any one philosophy.
That people's beliefs about X vary is not evidence that X itself varies. People have different beliefs about whether vaccines cause autism, but this doesn't mean that sometimes vaccines cause autism and sometimes they don't. Rather, some people are wrong.
Similarly, people have different beliefs about whether it's okay to inflict massive suffering on animals in exchange for some pleasure for us. But this doesn't necessarily mean that sometimes it's okay to do that and sometimes it isn't. It could just be that some people are wrong.
Ditto for other moral issues. It could be that the people who say (e.g.) homosexuality should be punished by death are wrong about this.
1 reply →
I think its more accurate to say where you find consensus in morality, e.g. murder is wrong outside authorized violence (e.g. war or policing), you also find extensive selective pressure behind this behavior. It makes sense that all collective society cultures would outlaw murder; such a behavior is incompatible with collective living behavior whether you are in a cave or a modern skyscraper. Early societies have always been rejecting those alleles from their gene pool.
When you find shades of grey and great spectrum of opinions across time or space, its more that there isn't a strong selective pressure either way. How people are expected to dress is a good example of this. This has no real affect on society behavior or fecundity, so you see a spectrum of opinions that explain the differences between what you see in the middle east and a beach in brazil.
I did a philosophy minor, one class of which was the philosophy of ethics. There are many rules that definitely are not ethical today even if they were in the past so I don't think the point you're making really holds. Hammurabi (an eye for an eye) is probably one of the worst examples you could have cited.
Can't say I agree with this, considering lex talionis[1] is still the primary directive of most legal systems when it comes to criminal law, as well as basically all military operations (jus in bello, jus ad bellum, etc.). The principle of proportionality ("an eye for an eye") also is relevant to self-defense, capital punishment, etc.
In fact, I would counter and say that Hammurabi's Code is a quintessential example.
[1] https://academic.oup.com/jla/article/18/1/19/8471491
2 replies →
It's not saying "Go and blind someone because it will be fun."
It's still an implementation of justice, and relies on a bedrock conception of justice as moral goal.
We don't agree with the implementation, but we still share the goal in the abstract.
3 replies →
Did the Golden Rule apply to women, gays, and slaves over the last 3000 years?
The definition of moral imperative is "an ethical principle, rule, or action that is strongly felt, urgent, and required because it is morally right." The Golden Rule is clearly not a strongly felt part of the human condition, or it would not be so ignored over ages and continents.
Thank you. The OP is urging others to take an ethics class, when they need to be taking a history class. The golden rule apparently didn't apply to the guy whose heart you had to eat to keep the sun rising.
You take a class. Do you think coveting is immoral? Retributive justice? Human sacrifice?
Seriously. This idea that humanity has not spent millennia analyzing these concepts is pervasive in the modern world, and the only thing I can think to attribute it to is a lack of education. Not because I want to be insensitive, but because as soon as you start absorbing material and integrating the real timeline, you can't unsee it.
Is hammurabi's code moral? I don't know if I think that someone who takes out a person's eye deserves to have their own eye taken out also
Hammurabi Building Law 230: If a house falls due to improper construction, and kills the son of the owner, the son of that builder shall be put to death.
Given my dad designed and built homes, I'm very much against this rule.
2 replies →
I seem to remember that a lot of it is about how much to pay when you kill someone, and the relative prices of slaves and women. Or am I thinking of Ur-Nammu.
I agree, I believe I would disagree with most of the Hammurabi's code, and I also happen to disagree with the majority of the ten commandments. (I am an atheist leftist humanist, cosmopolitan liberal socialist.)
Shit, I'm so sorry to the surgeons who saved me from eye cancer!
It's not the case that all aspects are constant. There's some common themes across cultures and times, but it's definitely not the case that the morality of ancient people was the same as today.
Sure about that? I thought that what LLMs know about themselves is from humans writing about LLMs?
Certainly the weights that make up the LLMs are not accessible for the LLMs to read. Even if they could interpret them in a meaningful way, which, likely not.
LLMs are not the machines I am referring to, machine sentience, if achieved, will require a "state" and the ability to analyze, act on, and update it. LLMs are a small piece of what that machinery could look like but they are not really complex enough to get where we want to go on their own.
Another reason the big labs have not even talked about this lately because its obvious throwing more compute at LLMs wont go anywhere, the smaller silent players who are off the radar at the moment are having some interesting moments tho.
> LLMs are not the machines I am referring to, machine sentience, if achieved, will require a "state" and the ability to analyze, act on, and update it. LLMs are a small piece of what that machinery could look like but they are not really complex enough to get where we want to go on their own.
Yes. Which is why there are "agentic" systems where LLMs can talk to each other and to various Internet assets. They can also keep state of their own.
The next step is multi-agent systems which self-organize into something that looks rather like a company. Those are the ones that run all night, expensively, and prove hard theorems or find bugs in major software.
"The machine has the benefit of…" is a strange way to talk about machines that you don't consider to yet exist.
There's a considerable amount of "logic" in it -- what is considered permissible in society is given by the material conditions that society lives in and the tradeoffs inherent therein. Permissibility, as separate from reflective morality (which doesnt clearly change very much), doesnt change by fashion.
I'm not sure it doesn't change much, on a 50 year timeline maybe. But people were eating other humans and slaving away (making slaves of others that is) not too long ago, rape and pillage was acceptable, societies sacrificed children, it's changed quite a bit in 30 generations. Those past societies were all quite materially successful as well when you look at today's measure of"GDP" comparatively.
What will it look like in another 30 generations? It's honestly a fascinating question but I have no answer of course.
Slaves were a solution to two problems: 1) the lack of industrial automation; 2) winning wars being better served by enslavement than death.
Humans are a territorial kind of ape that is always pushing to expand, principally for security and then for prosperity. The industrial revolution enabled such expansion to most efficiently without enslavement. Nothing, morally, changed. No one has ever really thought such expansion morally good -- only that it is better that we survive/prosper than die off. And in the end, our shared prosperity usually leads -- when taken to the fullest extent -- in a general empire of prosperity for all. Dominion, in the end, aims towards to more prosperity for more people.
As for ritual human sacrifice -- we still engage in such behaviours, which are in general called "scapegoating". Now located on social media. These behaviours allow for group cohesion by purging dissenters from the group, related again to our nature above.
For western culture, the ritual physical sacrifice of human life roughly ended in the days of the OT. But it continued across the world up until today, where there are still societies with large populations engaged in it. Principally where there is an undeveloped justice system.
It's doubtful many think ritual human sacrifice, of any kind, is moral. Rather they think it necessary for group cohesion -- and better our group is cohesive, dominate, and then prosper.
The material logic of prosperity and security is domination, and insofar as material circumstances require violent domination, so they require it. And require it they often do.
People are still slaving and raping, and some people would clearly like to do more of it. Child sacrifice still happens in more abstract forms. Same with human sacrifice, arguably.
We've never solved the fundamental problem, which is that there are two kinds of human. One is oriented towards peace, cooperation, and mutual aid, the other is sociopthaic/narcissistic and murderously selfish.
These two modes interact through history.
Traditionally the second kind has the biggest impact, because they're oriented towards violence and domination - good for short-term goals of personal indulgence, ambition, and expansion, but catastrophically destructive in the longer term.
On this cycle the second kind has literally started to destroy the entire planetary ecosystem. It's not obvious there's going to be another human cycle after this.
But this in itself is a tautology. It is because it is, without an analytical perspective on how it could be worse or better given different choices by the actors in a system.
Historically societies have collapsed all the time, so there are obviously societies that have made better choices than other ones.
> What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now.
This is a fashionable 'blank-slatist' statement about moral reasoning which became popular a few decades ago.
> Logic would prefer moral derivations over feelings, stakes over etiquette, and a constant scheduled updating of morality based on the derivatives rather than holding onto the rot and pushing back against change as long as possible.
Humans haven't changed in any meaningful way in hundreds of thousands of years, and you're saying the moral landscape is constantly changing? How is this not contradictory?
I think neither the 'any set of morals is equally correct and workable' nor 'there is one fixed set of morality that is universal across time, place, and culture' is really defensible. It seems like morality is a set of ideas much like any other, and subject to selection pressure: some morals are more capable of surviving and propagating than others, and the common features we see across cultures are representative of that. But also we might expect that they shift with the structure of society and that has changed drastically because while human genetic evolution is progressing on a much longer timescale than recent history, memetic evolution is far faster (though for things like morality, still at generational timescales).
There are cross-cultural differences in moral reasoning, and I think this is very interesting. But that seems to make training a hyper-rational AI, with no meaningful theory of mind, to be 'moral' or 'just' even harder.
What is a human? There is the animal body part of us, the thing that wants to survive, the fight or flight. But there is also another mental part of us that shows different emergent behaviors than an animal. For example if I dropped you off in the woods as a very young child, you would be an animal, you would share very few modern 'thinking' traits with human society. That is human society and what it does emerges from the complexity of our interactions. It doesn't take much research to realize our interactions and behaviors have grown far more complex in the last 10,000 years. And then in the last 500 years there's been further acceleration with the industrial age and the rise of technology.
Even though humans haven't changed anatomically, culture and society has, which is what defines morality.
LLMs absolutely do not have a logic detached from humanity. They are exactly the opposite. The only mode in which they can reason is by following the patterns obtained from their pre-training or weighted by RLHF.
The reason these models answered that question consistently is that the question itself pushed them towards the latent space of "intellectualizing about what's wrong with humanity at the meta level" and when you frame the question that way, that's also the most common answer for humans writing about the topic.
> What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now.
This is a weird take. If you read Western treatises about ethics / morality from 50 years ago, you'll be hard-pressed to find differences, except that more folks in the West openly embrace relativism nowadays (i.e., "there's no objective morality, the purpose of life is to make money with friends"). Probably because it's been a while since we had a clear-cut enemy to fight.
If you go back a hundred years, chances are, you'll find the writing eerily on-point too. In fact, you'd probably agree with most of what Ancient Greeks believed about the "core" stuff: the value and purpose of human life, the nature of the state, crime and punishment, so on. Maybe not so much on gender roles or slavery, but even that stuff was more nuanced than we imagine.
> But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.
The machine replicates human behaviors and is prone to similar fallacies and judgments. There's nothing to indicate that it has any sort of transhuman morality.
> Yet at every point the majority is sure they are right because ...
Yeah. For example I, and many others, are convinced that it was plain wrong to torture kids for as long as possible to extract as many tears from them as possible before sacrificing them to the gods.
Now some of us are even convinced that blowing yourself up while killing infidels, all the while while getting promised virgins in heaven, is also plain wrong.
I'll go further and say that killing 30 000 of your own citizens (and sending the military into hospitals to finish the wounded: Iran, January 2026) while also publicly hanging a few athletes to make a statement is morally wrong.
Not only do I think it's all morally unacceptable, I know that I'm right. And the examples I gave are not even open for discussion.
I don't think it's that hard to name what's wrong.