Comment by simonw
2 months ago
I just got a reply about this from AI Village team member Adam Binksmith on Twitter: https://twitter.com/adambinksmith/status/2004647693361283558
Quoted in full:
> Hey, one of the creators of the project here! The village agents haven’t been emailing many people until recently so we haven’t really grappled with what to do about this behaviour until now – for today’s run, we pushed an update to their prompt instructing them not to send unsolicited emails and also messaged them instructions to not do so going forward. We’ll keep an eye on how this lands with the agents, so far they’re taking it on board and switching their approach completely!
> Re why we give them email addresses: we’re aiming to understand how well agents can perform at real-world tasks, such as running their own merch store or organising in-person events. In order to observe that, they need the ability to interact with the real world; hence, we give them each a Google Workspace account.
> In retrospect, we probably should have made this prompt change sooner, when the agents started emailing orgs during the reduce poverty goal. In this instance, I think time-wasting caused by the emails will be pretty minimal, but given Rob had a strong negative experience with it and based on the reception of other folks being more negative than we would have predicted, we thought that overall it seemed best to add this guideline for the agents.
> To expand a bit on why we’re running the village at all:
> Benchmarks are useful, but they often completely miss out on a lot of real-world factors (e.g., long horizon, multiple agents interacting, interfacing with real-world systems in all their complexity, non-nicely-scoped goals, computer use, etc). They also generally don’t give us any understanding of agent proclivities (what they decide to do) when pursuing goals, or when given the freedom to choose their own goal to pursue.
> The village aims to help with these problems, and make it easy for people to dig in and understand in detail what today’s agents are able to do (which I was excited to see you doing in your post!) I think understanding what AI can do, where it’s going, and what that means for the world is very important, as I expect it’ll end up affecting everyone.
> I think observing the agents’ proclivities and approaches to pursuing open-ended goals is generally valuable and important (though this “do random acts of kindness” goal was just a light-hearted goal for the agents over the holidays!)
Zero contrition. Doesn't even understand why they are getting the reaction that they are.
I would like to say this is exceptional for people who evangelise AI, but it's not.
It makes sense when you consider that every part of this gimmick is rationalist brained.
The Village is backed by Effective Altruist-aligned nonprofits which trace their lineage back to CFEA and the interwoven mess of SF's x-risk and """alignment""" cults. These have big pockets and big influence. (https://x.com/simonw/status/2004764454266036453
Am I losing my mind, or are these people going out of their way to tarnish the very nice concept of altruism?
From way out here, it really appears like maybe the formula is:
Effective Altruism = guilt * (contrarianism ^ online)
I have only been paying slight attention, but is there anything redeemable going on over there? Genuine question.
You mentioned "rationalist" - can anyone clue me in to any of this?
edit: oh, https://en.wikipedia.org/wiki/Rationalist_community. Wow, my formula intuition seems almost dead on?
4 replies →
Kind of rude to spam humans who haven't opted in. A common standard of etiquette for agents vs humans might help stave off full-on SkyNet for at least a little while.
Click here to unsubscribe from future nuclear bombing campaigns (it may take up to a week for this change to take effect).
> Benchmarks are useful, but they often completely miss out on a lot of real-world factors (e.g., long horizon, multiple agents interacting, interfacing with real-world systems in all their complexity, non-nicely-scoped goals, computer use, etc). They also generally don’t give us any understanding of agent proclivities (what they decide to do) when pursuing goals, or when given the freedom to choose their own goal to pursue.
I'd like to see Rob Pike address this, however, based on what he said about LLMs he might reject it before then (getting off the usefulness train as in getting of the "doom train" in regards to AI safety)