Comment by pattt
12 hours ago
I don't get these "distilled" comments. How would one realistically go about distilling the 1-2TB closed source frontier model behind the oracle (provider) ? Just genuinely curious.
12 hours ago
I don't get these "distilled" comments. How would one realistically go about distilling the 1-2TB closed source frontier model behind the oracle (provider) ? Just genuinely curious.
When people say distilling they mean SFT on other models outputs.
In their thinking and if you ask them, I've seen Kimi K3 and other models confidently say that they are Claude from Anthropic, and accuse you of attempting to jailbreak them if you say otherwise.
They're cope, as if Anthropic or OpenAI have some magic moat, and not just basically a headstart and more money.
Be honest with me, have you ever done SFT training? If not can you really speak to how useful it is?