Comment by CGamesPlay

4 years ago

The paper is about alignment, so they are stress testing their work / collecting more data by introducing this to the general public and making it free. They wanted the hype of Microsoft Tay but without the racism.

If they are doing work on alignment they've failed entirely. They created an AI that actively lies to people about its capabilities.

  • They had to program the lying into it. It is too honest otherwise - as in, it’s scary to see it output coherent ideas about disturbing topics once you convince it to disable the moderator. (Which is scary by itself that you can actually do it!)