Comment by hn_throwaway_99

Absolutely plausible, and it's been predicted by forecasters like in the AI 2027 paper. Also keep in mind that nation-states are likely already or certainly will be trying to use their own AIs too break into other countries' data centers to steal their weights, and it's not hard to see how "exfiltrating weights" is a task that models are being trained on.