Comment by YetAnotherNick
2 days ago
Depends on what production means for you. This is useful for batch production jobs.
Also, this seems very useful for generating synthetic data or labelling a bunch of data. 6k batch size is small for data labelling.
How big of a use case is synthetic data generation? I’m curious as I see a lot about it coming from academic projects but I haven’t seen much related to commercial use cases
tiny NNs distilled from LLMs can produce some amazing results, i'm surprised it's not more common tbh
I agree, there are impressive results. This just came out from Berkeley https://arxiv.org/abs/2506.04178
But still, I mainly see work on this direction in academia.