Comment by monkeydust

The way I see this (I havent played around with Jev or layla the OSS version) is that classifiers have always existed and a recognised tool in the ML world. But, the norm is that one needs to not only know what to classify as, but determine what weights to use to classify the input.

Jev came in, and added that magic of "you dont need to train your classifier or determine the weights" if you dont want to, and just get the classified answer out. I think that's what is making people see this with a glitter in their eyes.

>Classification models have been around for a while.

I'm still trying to catch-up on the Jev stuff, but my understanding is that it's basically just a more efficient LLM when all you want is the LLM to produce a classification.

There's more to it, of course, but it's not just "generic" classification ML because it accepts arbitrary inputs and can produce probabilities over arbitrary classes. Not saying this is the first time people have done this, but typically classification tasks are more static and limited.

In the same vein, it's also not just an LLM with structured outputs (which have been a thing for a while) specifically because that is a very inefficient way to approach classification using this kind of architecture. Jev models are much more performant because of how limited they are compared to a full LLM.

So when you want an LLM, but you only really need this kind of classification from the LLM, then Jev makes a ton of sense. This makes sense for me, since I've definitely used LLMs for this kind of classification work and, even then, it kind of felt like using a jackhammer to place some nails, etc.

Happy to be correct, though.

The Jev model is economically, but also in terms of compute, a much more efficient model. A normal LLM goes token by token, each token in a separate step. Whereas Jev just returns all the results the first round. So it is much better at classification than LLMs.

Compared to traditional ML classification, Jev works without training, like a LLM.

It reminds me a bit of what Ansible got right: user communication. The underlying tech may have existed for a long time, but the genius is presenting it to a regular developer in a way that reads "yes, even you can understand ML, just using a little JSON". The contribution of that should not be understated, as has been clearly evident recently.

Previous classification models need to be trained on the specific question/choices you are trying to output. Jev doesn't need to be retrained for every choice set provided.

LLMs can act as classifiers but they still have to generate text output in the form of a JSON object. This means they have to generate every single curly bracket, quote, command, etc. This turns out to be pretty expensive. On the other hand, Jev uses a different decision head so it doesn't generate text output at all, it outputs logits *only* for the choices provided. So it completely avoids the need to generate text at all, which means no malformed JSON and it's much faster as a result.

Finally, Jev also provides confidence scores that are actually reliable (not made up like LLMs).

It's appealing not having to fine-tune separate model for each use case

So you have more flexibility to get on with building, evolve your business logic etc

We know how useful classification models are, but massive pretraining is even more useful. So useful that it's tempting to just use a generic autoregressive model for everything, rather than trying to train a specialized classifier.

I think specialized models of all sorts, if it were possible to train them on equally much data, would outperform general models. In cases where we can have the data, like in self-play in games, they clearly do and have done so for a long time.

But it's a lot of effort to train specialized models. I think it's too early to tell if Jev delivers on its promises to give the best of both worlds, but if it doesn't, we'll have to make it work. It's just the right thing to try.

For a while is the keyword. It’s just vibe coders have just discovered the classifiers

It’s because it’s practically useful and enabled things that were impractical previously.

I think the reason is in the general ballpark of people throwing LLMs at a huge variety of problems and being too slow disqualifies them from a bunch of things.

Now there's a new training-free thing that is fast enough to be useful on a new class of problem.

If you have a little data and can ask a frontier LLM to train a model, you can probably beat it on average for a specific task.

But... This is the case with LLMs too.

It’s simply not understanding. In terms of latency, LLM processing for large fact-based decision-making was incredibly slow. The real unlock here was speed in decision-making.

Feels like astroturfing.

I think it’s timing. So many devs trying to squeeze their subscriptions, build more tooling to throughputMaxx. 6 months ago, i speculate it launches pretty flat.

Could you link to some of these classification models that can be used as versatile and perform with similar quality, speed and cost?

Classifier models are extremely niche and trained for a singular purpose. A utility classifier that you can one-shot on almost any topic or need is a dramatically different beast.

Is it truly useful or accurate or beneficial? To be seen. But it's the idea that has everyone so captivated. An expert system that is an expert at most everything is a lot more useful than an expert system that is an expert at choosing a bar of soap, for instance.

[flagged]

Jev is creating a sort of identity crisis for me, because the number of absolutely clueless folks parroting the classifier thing is the first time I've seen this sort of mass psychosis in CS upfront.

Like even 5 minutes of tinkering captures why this isn't anymore like BERT or any past classification model than ChatGPT is like those old Markov Chain generators, yet folks cannot shut up about how this is nothing new.

Absolutely scary and makes me wonder how much of the field is just people super confidently discrediting otherwise promising/interesting directions for development for a cheap dunk!