Comment by electroglyph
3 days ago
tiny NNs distilled from LLMs can produce some amazing results, i'm surprised it's not more common tbh
3 days ago
tiny NNs distilled from LLMs can produce some amazing results, i'm surprised it's not more common tbh
I agree, there are impressive results. This just came out from Berkeley https://arxiv.org/abs/2506.04178
But still, I mainly see work on this direction in academia.