Comment by tym0
3 days ago
Yeah I've done more research and that's what I meant in the edit you replied to.
But that's not the full reasoning token context, just a snapshot of the latent state at the end of it, no?
Have a look at the gemini ones they're pretty small.
There was an attack that convinced models to leak the contents of the reasoning tokens, and it came back as text that matched the length of the encrypted data very well. So it's probably still tokens.
Looping back latent state isn't that easy. The hidden data is the entire contents of the KV cache which can be massive. I don't think any provider is trying to loop the KV cache through the client, and neural compressions of the reasoning would be lossy / an advanced technique that is still firmly in the realm of research papers, as I understand.