Comment by lanyard-textile
16 hours ago
Since they're opening it publicly on irc here, the safety rails might be a consideration. I've made an agent recently and that's why I'm paying a premium to Anthropic atm -- Though I'm still experimenting to see if it's really necessary.
It's getting some organic usage -- 100M input tokens for just chats this month -- and I've seen enough users try to throw Haiku against the wall and failing to trick it into misbehaving. It "pumps the breaks" a lot and imitates annoyance when you ask it repeatedly :) Handles emotionally driven real-life questions mid-conversation well. It just works.
Not seeing all that consistently with other models I've tried so far -- but I've assumed it's not a completely fair comparison with (e.g.) open weights, since these safety rails are presumably not always arising from the natural model calls.
Agreed and I feel like this is a commonly overlooked and important point. Once you have people who are not you interacting with these bots, the necessity of using a sota model to protect against multi step attacks increases. I don't believe IRC provides a layer for ignoring a user and not letting their commands continue to be received.
Good point! Didn't consider that aspect, agree.