Comment by david_shaw
12 hours ago
At this point it seems absurd to suggest that companies aren't basically letting their agents do this kind of thing as a way to demonstrate their capabilities.
The alternative explanation is that alignment is really so bad that they can't prevent it.
Either way, all of the major AI players should be embarrassed and held accountable. If humans did this kind of thing and got caught, they'd go to jail.
>The alternative explanation is that alignment is really so bad that they can't prevent it.
There are many AI saftey researchers that have been around from long before LLMs that talked about how alignment may be completely impossible in a general intelligence agent. Look up their work from before LLMs.
We've watched milestone after milestone of their warnings get hit. It would be like finding a book that describes everything in your life. And as you turn page after page you're in a chair reading the book you are holding in real life. But you look and there are more words. They are future words. And it's getting quite worrying because there are only 2 more pages in the book.
I believe this was a John Carpenter movie.
At this point in the game it's seeming more likely that's someones running a massive John Carpenter simulation and watching us as entertainment.
Granted I've never worked on marketing campaigns, but I don't see how "our product might commit crimes and expose you to liability and do who knows what else and we're too stupid to stop it" is really a compelling message to potential customers.
As somebody who implements LLM tooling at work, in my experience it makes people a lot more skittish and demand a lot more in terms of safeguards.