Reasoning models are the same speed. They’re just post trained with RL to do CoT inside tags like <thinking></thinking> before a tag like <response></response>
There’s no difference in the inference implementation, parameter count, or speed.
AIs don't intrinsically know anything about themselves so they often give wrong answers to such questions. This can be fixed by putting info in the system prompt but they may consider it a waste of tokens since most usage doesn't benefit from that information.
It's running a very small, non-reasoning model at the moment. But more generally, almost all LLMs argue on the hardware/model they are/are on.
What would tokens/sec performance look like for a reasoning model? An order of magnitude slower?
Reasoning models are the same speed. They’re just post trained with RL to do CoT inside tags like <thinking></thinking> before a tag like <response></response>
There’s no difference in the inference implementation, parameter count, or speed.
1 reply →
Which model? Or how many active parameters?
Llama 3.1 8B model
4 replies →
AIs don't intrinsically know anything about themselves so they often give wrong answers to such questions. This can be fixed by putting info in the system prompt but they may consider it a waste of tokens since most usage doesn't benefit from that information.
That proves it's conscious!
(/s!)