Comment by Nevermark
8 days ago
If it is a Turing Test, it’s a test if our intelligence and we failed.
The internet is a wild west of privacy, security, social and ethical holes an army of grifters routinely drive through. And in the case of some famous big firms, leverage and magnify at scale.
That is bad enough.
But setting up a horde of intelligent beings, so that those holes are their critical path to survival, is like pouring poison into the water supply to see what happens.
Can’t argue with “interesting”. It is that.
I mean there's no reason that the AI would need to act maliciously, thought it would be possible. I was thinking more along the lines of pre-pay for a year of server and tell the AI it has to start a business or something to pay its own bills.
If those prompt limits were reliable borders it would be very cool.
But that would mean the alignment problem was solved. When survival depends on watering down ethics, and there is a long slippery slope of ethical wells available to plum, depending on prompts for safety sounds risky.
Is that malicious? I don't think that would be considered malicious. We don't consider it immoral for starving people to steal food.
But, I am fascinated by the idea too! I just think it is a terrible idea (despite being almost certain to happen).