Comment by mikeocool
1 day ago
If you have to classify user input as they’re inputting it to provide a response — so it can’t be batched - 2 seconds could potentially be really slow.
Though LLMs sure have made creating training data to train old school models for those cases a lot easier.
Yeah, that’s what I do: use LLM to help make training data for small models. It’s ao much more efficient, fast, and ergo, scalable.