Comment by big_toast
13 hours ago
It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing.
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
CEO here - that is right!
I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable).
But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
Small request, can we get an explanation of the naming of "noul" in the docs[0]. I tried googling, and searching the docs and didn't understand why it was called that.
(I'd also argue something like p_yes or just probability might be a simpler name, but I'm sure there's a better reason behind Bernoulli maps).
[0] https://docs.typesafe.ai/primitives/noul#noul
Im going to guess bernoulli
1 reply →
here is how I attempted to explain it to my company's AI group chat, is this roughly accurate?
"instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel
so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K.
all of this while still being instruction-tuned (!!!)"
It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
very accurate!
the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing
Hi - first congratulations, System One looks really promising.
The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
I was confused at first too, but it makes more sense when you read about their primitives. E.g. https://docs.typesafe.ai/primitives/noul
The demo is showing System One producing its output in parallel very quickly and for little cost compared to an LLM generating its answers token-by-token. The "noul" type is used to evaluate a yes/no question and return the probability that the answer is yes.
So this demo is showing System One offering much more nuanced responses and specific probabilities compared to an LLM's more crude responses (e.g. LLM shows "true" or "false" compared to "0.9" or "0.07" probabilities that the answer to some question is true).
I think that's to demonstrate its speed
I see this super interestingly as the "subconscious" to the llms "conscious" for lack of better terms. I'm super interested in this for broad and rapid decision making in the context of consumer agents so will be signing up for sure.
1. I am extremely on the same page 2. I do think that subconscious is not only much smarter than we give it credit for, but also much more robust than the "jagged frontier" of current LLMs
(shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)
For many day-to-day computing use cases, Jev seems far better suited than an autoregressive language model, if for no other reason than it is not wasting compute thinking about anything other than how to spit out a decision.
Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
architecture is close to the chest for now, but we have talked about writing a paper
I don't want to shill my blog too much, but I will say data is probably far most interesting than architecture: https://www.completeskeptic.com/p/the-bitterest-lesson
It's unclear if the context extends as the conversation grows?
> the model takes as input a state (structured text? not sure if multi-modal)
Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure.
e.g.: https://docs.typesafe.ai/primitives/advanced#structured-inst...
> not sure if multi-modal
just JSON... for now :)
> outputs the question's answers as appropriate
correct!
I assume this isn't really for consumers/individuals currently? Kinda feels like an improved magic 8 ball.
I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
> I assume this isn't really for consumers/individuals currently?
Unless they're hackers, no. It's not really a chat interface, it's meant for consumption by machines and composing into higher level systems (pairs great with LLMs).
> Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
We're going to release some more info on evaluations over time, and yeah, join the waitlist! We offer faster access in exchange for good memes