Comment by lubujackson
14 hours ago
After much fumbling around with prompts and evals, this is exactly how I am using LLMs in production, to narrowly make choices and return structured data. Any deterministic work gets pulled out of the prompt and my goal is to narrow the model output to be as clearly defined and as minimal as possible.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
we hope so! the bigger hope is to not just eat LLM market share, but to allow for people to use AI much more in the inner loop of software
I'm sure you've thought of self-driving. How does the model work in that space?
Great question! Yes, this works much like the doom player. Sensor data (LIDAR, velocity, etc.) becomes the state. You use the score primitive to operate the controls ("What level of braking should be applied" 0: None, 1: just slightly slowing down, 2: there's a suspicious cat on the side of the road you don't trust, ...
Full disclosure, I am not they :=)
But the real problem in self driving isn't the decision making but object description. That is, computer vision if with cameras.
Decision making isn't that of a bottleneck I suppose.
This smells like a tool a more broadly capable LLM would take advantage of extremely well.
Curious what your use case is if not confidential.
Not confidential, but not super relevant, as this is something I have learned the hard way over the past year across various projects.
A lot of people have become prompt maximalists, asking for complex multi-part solutions or dynamic workflows in a single prompt. You can get this to work sort of reliably with frontier models, but without much confidence or clarity where things might break in practice. My goal is to strip out as much determinism as possible from prompts so the LLM only needs to handle a narrow, well-informed decision, like "Pick one of these three things" and build around the answer. Sometimes you need to fill out a whole JSON payload and LLMs really actually suck at manipulating and adhering to JSON. They do ok now because labs have put in a ton of effort on making harnesses play nice with structured data. But it comes at a high token and context cost because under the hood I suspect the model is churning invalid text repeatedly until it gets around to passing some internal validation.
Example I have worked: Personal delivery app, that tracks packages from various senders using incoming emails.
I am using the single prompt approach with GPT5.4, which is free, but it’s not reliable. Using Jev I’d decompose the prompt into a bunch of smaller questions, then I’d combine the answers in software. I’m super excited to try Jev out.