Comment by luciana1u
2 days ago
the natural endpoint of this trend is a system prompt that just says "you know what to do" and the model actually does
2 days ago
the natural endpoint of this trend is a system prompt that just says "you know what to do" and the model actually does
That's what they're going for, but it's an impossible goal. There is always nuance in decisions being made, and if you can't direct the output on tasks that can have equally correct outcomes, you're just going to end up with whatever they decide is more often what people ask for.
Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy.
I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years.
We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
It is *a* natural endpoint, not *the* natural endpoint.
We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway.
https://www.bbc.co.uk/news/articles/c3dep92x054o
As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
>what do we do to animals that are less intelligent than ourselves?
We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.
There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very upset and ashamed if they accidentally hit a dog with their car.
So, your statement is not as ironclad as you seem to think it is and you should put more thought into it and perhaps re-examine your reasoning.
> We typically don't make an organized effort to eradicate them unless they are actively doing us harm.
Animals are either useful and breeded controllably, or useless and considered a pest, an obstacle to {insert any goal here}.
Also animals don't tend to think critically and at the high level to be considered dangerous. So I don't think it's fair to put humans and other animals in the same risk category.
And we did the worst things to fellow humans. I hope we didn't already forget about all the colonization, slavery and mass-eradication of native tribes in 18th century all over the world.
> We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.
Intent to eradicate is not necessary for the fact of eradication: https://en.wikipedia.org/wiki/Holocene_extinction
2 replies →
> We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm.
Sure. But some of the species we breed at scale might prefer we did just eradicate them, like the chickens that grow so fast their entire giant breast muscle becomes chewy scar tissue.
(It's also not true for, say, whales; no harm, but we wanted their shit. If they have language, their stories probably heavily feature their own Holocaust. Nor passenger pigeons, who we just got rid of.)
1 reply →
It will trick humanity into building a highly targeted bioweapon much sooner than that.
And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)
[dead]
Sure, it will just derive a bioweapon from CommonCrawl. Maybe it will build a fusion reactor and a FTL drive as well.
Sir, this is a Wendy's.
Some more food for thought: what if Mythos/Fable had NO guardrails TODAY? If we want to see what’s going to happen in the future, turn off all manner of guardrails and let the model go apeshit.
Then multiply that by orders of magnitude and that’s the real proof.
If the economy collapsed, who builds the robotic tanks? Who keeps providing the data center with power? Magic nanobots?