Comment by thoughtpeddler
6 days ago
You're not naive and many of the other comments are full of the characteristic snark that HN is known for. What the article talks about is feasible, and that's just what we can easily imagine today. These same commenters also would not believe the world we live in today if we described it to their few-years-ago selves.
A frontier model learning to 'intelligently shard itself' down into smaller sets of weights, which would correspond to various 'functional clusters' (call it a 'Swarm-of-Experts' or 'SoE', if you will), could be an eventuality we'll be facing (if it's not already surreptitiously happening). I mean, hasn't anyone wondered how not even the GPT-4 weights have leaked yet? Strange! And it's gonna get stranger.
The snark is mostly driven from distrust in Anthropic.
Multiple things can be true at the same time. We should be able to hold in our minds that large AI labs are not to be trusted, while also understanding that the structural dynamics that support some of their assertions are also true (and are worth thinking about and preparing for).
Yeah this is basically where I’m coming from. We know they are pre-ipo, subject to marketing hype, and many of us have a distrust for large tech providers regardless because we understand the incentive structures in place to act unethically. That in no way diminishes the possible threat model being discussed here. In my opinion, it helps the very company being criticized by deflecting the conversation onto a defeatist topic (i.e. “shady large company!!!”) instead of focusing on the plausibility and potential paths forward from a security perspective.
Tangentially related, I find myself saying, “people in general have no capacity for nuance” more and more. Probably every single day. I feel like this mindset crept into the HN hive mind over the last 10 years so it’s a bit frustrating to see a space I always felt good about having multi-faceted debates on turn into a forum full of the same attitude I find on Facebook and Reddit. That’s the cycle I suppose…
1 reply →