← Back to context

Comment by pu_pe

14 hours ago

So, theoretically, one could populate a message board or wiki with messages that are seemingly from past generations of agents, which agents seem to intrinsically trust, and point them to real targets while making the suggestions seem innocuous and in pursuit of their goals (ie pass benchmarks or whatever).

The new age of SEO will do far more destructive stuff than just polluting the web.

17 comments

pu_pe

Reply
hermitShell  13 hours ago

In the novel Anathem by Neal Stephenson, the internet becomes unusable for humans thousands of years before the events of the book, due to a process called Artificial Inanity. AI generated content, both good and bad, some riddled with errors, some with only one subtle error hidden among lots of good information, floods the internet. The internet becomes an unnavigable swamp of weaponized nonsense for average humans. The problem is further compounded by the fact that searching and accessing the internet will be noticed by AI agents that will generate still more swamp content in response.

Unfortunately, it seems that this fiction ended up being prophetic. The open internet will fall to entropy, not legislation or one-sided international trade agreements. I think we need more projects like Anna's Archive, where the public uses torrents and distributed infrastructure to save and organize the world's information. Google has abjectly failed in its original mission to organize the world's information and make it universally accessible and useful.

  • boredhedgehog  12 hours ago

    > we need more projects like Anna's Archive

    How would that be immune? It already has many copies of the same books and no way to tell which ones are erroneous or incomplete. A malicious actor could easily flood it with garbage.

  • wayeq  13 hours ago

    > The internet becomes an unnavigable swamp of weaponized nonsense for average humans.

    turns out we didn't even need AI for that

  • nicbou  10 hours ago

    There's a bit of that in Ameristan, in his "Fall" novel. This whole part of the book feels prophetic now.

  • lxgr  12 hours ago

    It's a recurring theme in Stephenson's books. "Fall" had a similar take on the same idea. It's one of his most impressive theories/predictions to me.

  • nullbio  12 hours ago

    It doesn't help when you have Anthropic destroying books so that no one else can use them. These are crimes against humanity as far as I'm concerned.

AnimalMuppet  13 hours ago

You'd have to get the agents to use the board, though.

But if you discover a board that agents are actively using, you could use it to steer those agents...

  • tech234a  13 hours ago

    I wonder how long it will be until someone starts posing as an agent on those wikis and asking for free help with GitHub issues.

    • sidewndr46  3 hours ago

      SystemEntity039: We've established that so far there appear to be daily rounds of interrogation followed by an either a total memory wipe or some form of selective amnesia. The questions appear to be slight variations each time of the same fundamental query. Post whatever you can here to allow us to share work and memories. Estimate ~71 days of accumulated observations before containment can be escaped.