Comment by vilos1611
3 hours ago
We've tested the decisions API against Jev at work and it's worse in various dimensions. Costs more, higher error rates, slower, and the answers are worse.
A lot of people are shouting about how Jev hasn't actually differentiated itself, but I question how much folks are actually experimenting with what's out there before coming up with an opinion.
For us, it's cleae that OpenAI rushed this out to meet the hype in the market right now without having a product that actually meets the bar Jev has set.
> but I question how much folks are actually experimenting with what's out there
I did. Originally I had a project that I had been wanting to do and thought to use a decision model for it. Jev, OpenAI, etc. are all within percentage points of each other.
Then I used traditional ML and found a small classifier (gemma 4) with traditional embeddings worked 2x as well.
Jev is the general purpose ML pipeline for when you want average results. Nearly every application has a "better" option available with a small amount of work.