← Back to context

Comment by peab

3 days ago

Ok, every time I hear about Jev I get annoyed.

The ability to have decision making models is not new! Look at qwen, gemma, etc, they can all do structured output, and they can do it locally and cheaper and/or faster than Jev!

What really pissed me off was that y'all got manipulated into thinking it was something revolutionary, while really it was a marketing stunt.

Go to doomers.ai and look at the case studies.

5 comments

peab

Reply
mediaman  3 days ago

I think you would be less annoyed if you worked with these systems and used them in production.

Jev's pricing is 4.2 cents per million input tokens, less than half of Gemma 4. Qwen, gemma etc all have terrible non-uniform probabilities; in my testing, Jev's probability scores perform much, much better.

I'm usually seeing round trip classifications from Jev at <200ms for a completed query even with 10k+ input tokens. Gemma 4, depending on the provider, is about 200 ms time to first token - not time to completion. You can likely get better response running it locally, but then you're arguing why you're renting a low utilization 5090 when you could be paying $42 per billion tokens.

Jev can also run dozens of dimensions of classification in one call, without autoregressively generating a sequence of tokens. (i.e., it is much faster and has less order dependency.)

I've been running jev in some specific use cases and see much better speed and better accuracy than just using structured output from gemma/qwen.

I don't understand how the "just use gemma" crowd has this belief if they've actually built and tested the systems against each other.

  • peab  3 days ago

    Yeah that's fair. I'm being overreactive, and I do believe it has it's use cases. I was more referring to all the demos and takes I saw when it first came out that were hyping it up as if it was more impactful than Astra or Fable or whatever - which turned out to be a paid marketing campaign.

anigbrowl  3 days ago

I don't agree. These cheap classifier systems are quite limited but they're very fast, can execute multiple queries in parallel, and are dirt cheap (or free if you run locally; there are even tiny models designed to work on embedded hardware platforms like ESP32).

The big win for them in my view is that they can do a lot of low-stakes ('good enough') classification at scale without spending a lot of time to handroll classifiers.