But zero-shot classifiers with this level of intelligence, world knowledge, ergonomics, cost profile, and ease of use are new.
I feel like good engineering doesn't just ignore those things, or at least it didn't before recently. Now I guess social media has added a pressure to reduce everything to a hot take.
> Ergonomics, cost profile, and ease of use are new.
Following AI from the academic papers side; jev really feels silly. They one-pass the LLM tranformer stack and tune the output network for a probability value.
(some clever pararellization optimisations to make it viable to offer as an api, since the normal kv cashing no longer works if you oneshot the tranformer)
The largest change is the packaging; An api with a tolken based pricing, and a schema to define the output structure for quick setup.
Previous projects would probably involve installing pytorch, running a converter script on Qwen, and write a fair bit of matrix math to change the output shape.
They almost certainly would perform worse than more specialized classifiers trained with less data. It’s kind of a paradox of generalization. I think there’s an interesting space where you use generalized models to generate ad hoc specialized classifiers.
Depends what you man by "more specialised". You wont train very good language understanding without alot of data. It probably uses the core tranformer stack from an LLM.
Classic classifiers are regularly just tuned general models; Training a CCN on ImageNet and tune it for cats and dogs gives better results than just training it on cats and dogs.
There is likley a small network used to tranform model output vector to probabilities, but that wouldn't be massive. Retraining that small network for specific task may beat jev; but that's bairly considered training by modern standards.
Expecting a strong zero-shot performer to perform worse in a low data regime?
That only makes sense if you try to rope in data previously used to establish the model's priors, but that wouldn't make sense in this context. That same additional data is what enables things like...
> use generalized models to generate ad hoc specialized classifiers.
An LLM is a zero-shot classifier with a large number of classes. All you need to do is establish what the output means and you can fine-tune an LLM final layer for this task if you like (and others have done). A student of mine did this as an exercise two years ago, and it was cool, but not publishable.
I agree with you on the "ease of use" business though. No one thought to make this sort of thing commercially available.
But there is no hot take here. Jev is not some new paradigm; engineering-wise, it is a trivial modification to the existing pipeline. That doesn't mean it isn't commercially viable.
No, all they had to do was come up with a quality post-training recipe, production inference stack that wouldn't fall over, GTM, documentation, schemas, etc. etc.
(also most signs point to this being LLaDA 2.0-adjacent so throw in solving some substantial mid-training)
I think it's 100% a hot take to call what they built trivial. Or at least it used to be.
There was a time when that kind of stuff was something between sour grapes and cluelessness about the gap between an idea and an actual commercial product deployed at scale, but now that's just weirdly normalized.
In fact, if anything I'm the weirdo for repeatedly taking issue with the way people are trivializing it ¯\_(ツ)_/¯
But zero-shot classifiers with this level of intelligence, world knowledge, ergonomics, cost profile, and ease of use are new.
I feel like good engineering doesn't just ignore those things, or at least it didn't before recently. Now I guess social media has added a pressure to reduce everything to a hot take.
> Ergonomics, cost profile, and ease of use are new.
Following AI from the academic papers side; jev really feels silly. They one-pass the LLM tranformer stack and tune the output network for a probability value.
(some clever pararellization optimisations to make it viable to offer as an api, since the normal kv cashing no longer works if you oneshot the tranformer)
The largest change is the packaging; An api with a tolken based pricing, and a schema to define the output structure for quick setup.
Previous projects would probably involve installing pytorch, running a converter script on Qwen, and write a fair bit of matrix math to change the output shape.
I'm kinda amused that it took this long though.
They almost certainly would perform worse than more specialized classifiers trained with less data. It’s kind of a paradox of generalization. I think there’s an interesting space where you use generalized models to generate ad hoc specialized classifiers.
Depends what you man by "more specialised". You wont train very good language understanding without alot of data. It probably uses the core tranformer stack from an LLM.
Classic classifiers are regularly just tuned general models; Training a CCN on ImageNet and tune it for cats and dogs gives better results than just training it on cats and dogs.
There is likley a small network used to tranform model output vector to probabilities, but that wouldn't be massive. Retraining that small network for specific task may beat jev; but that's bairly considered training by modern standards.
Expecting a strong zero-shot performer to perform worse in a low data regime?
That only makes sense if you try to rope in data previously used to establish the model's priors, but that wouldn't make sense in this context. That same additional data is what enables things like...
> use generalized models to generate ad hoc specialized classifiers.
An LLM is a zero-shot classifier with a large number of classes. All you need to do is establish what the output means and you can fine-tune an LLM final layer for this task if you like (and others have done). A student of mine did this as an exercise two years ago, and it was cool, but not publishable.
I agree with you on the "ease of use" business though. No one thought to make this sort of thing commercially available.
But there is no hot take here. Jev is not some new paradigm; engineering-wise, it is a trivial modification to the existing pipeline. That doesn't mean it isn't commercially viable.
No, all they had to do was come up with a quality post-training recipe, production inference stack that wouldn't fall over, GTM, documentation, schemas, etc. etc.
(also most signs point to this being LLaDA 2.0-adjacent so throw in solving some substantial mid-training)
I think it's 100% a hot take to call what they built trivial. Or at least it used to be.
There was a time when that kind of stuff was something between sour grapes and cluelessness about the gap between an idea and an actual commercial product deployed at scale, but now that's just weirdly normalized.
In fact, if anything I'm the weirdo for repeatedly taking issue with the way people are trivializing it ¯\_(ツ)_/¯
3 replies →