Comment by alphazard
12 hours ago
There's a whole lot of information on this page that doesn't tell me anything about what this actually is. Can anyone spell out what the architecture is here?
They claim it's not an LLM, which I read as "not an auto-regressive token generator". I assume they are still using a transformer, otherwise they would be talking about the thing that's not a transformer, instead of all the fluff on the linked page. But they emphasize parallel generation, so is it like a text diffusion model?
Sounds like its essentially a generalized zero-shot classifier that takes and option set at runtime and works on unstructured inputs.
you pass in your "prompt" and options (described in natural language) that it can respond with, in addition to your input. it gives back that option set with a probability assigned to each one
yes and can do many of those in parallel
I would guess a tiny stripped down text diffusion model. It only has 32k context, and for choice mode it can only select from 10 choices.
> and for choice mode it can only select from 10 choices.
Rip there goes my excitement. I have a task that something like this would be great for but the list of options is a zero or two larger than that xd
They said that it works with up to 255 options.
you can still chain them