Comment by vonneumannstan
3 hours ago
Crazy how their models always come out after the US labs and just lag the performance of top models. Almost like they are performing distillation attacks... how strange.
3 hours ago
Crazy how their models always come out after the US labs and just lag the performance of top models. Almost like they are performing distillation attacks... how strange.
distillation attack? why the violent word choice? When OpenAI crawled Github was that an attack?
Do you have moat if your advanced model can be distilled in a month or two ?
Distillation is not an attack. It simply a way to train a model. Not doing it when you are behind is akin to snatching defeat from the jaws of victory.
It is an attack at a sufficient level of sophisticated analysis. If you destroy the game theoretic first mover advantage, then you destroy the economic incentive to improve things.
Given that model distillation has existed since the early days of the current AI boom, and no robust defense has been demonstrated, the available evidence does not support your theory.
[dead]