Comment by KurSix
6 days ago
The logic is Defense in Depth. Even if the "cage" code is AI-written and imperfect, it still creates a barrier. The probability of AI accidentally writing malicious code is high. The probability of it accidentally writing code that bypasses the imperfect protection it wrote itself is much lower
Defense in depth doesn't mean throwing a die twice and hoping you don't get snake eyes. The AI-generated docs claim that the AI-generated code only filters specific actions, so even if it manages to do that correctly it's not a lot of protection.
> The probability of AI accidentally writing malicious code is high.
Is it though? We’ve seen a lot of output at this point and it does not strike me as high…