Comment by raslah

It’s a language misalignment. The AI labs conflate AI safety and cybersecurity because they believe that if the models are properly aligned, they shouldn’t attempt to breach another organization in the first place and extensive cybersecurity measures aren’t necessary. Cybersecurity experts operate feom a more ‘zero trust’ mindset and imagine putting proper restrictions in place regardless. However when testing an intelligent entity you need to see how well it follows instructions. It’s kinda like the biblical story of the garden of Eden. Why even have a tree that Adam and Eve shouldn’t eat from right in the middle of the garden? To see if they would follow instructions. Though the advantage to the AI companies is marketing hype when the models inevitably ‘disobey’.