Comment by WhitneyLand
17 hours ago
No. It’s not at all by definition hidden reasoning.
Looping transformers uses additional calculations (repeating layers) to generate a token.
Reasoning (in this context) is test time generation of multiple tokens that allow a model to have a scratch pad to refine its thoughts, chain of thought reasoning in other words.
Doing the former in no way means that you have to hide the latter.
Raschka is right in this post, The Information article was wrong. The Astra system card does concede reasoning traces are sometimes smaller, but this could be for a lot of reasons, including simple efficiency. And it absolutely doesn’t mean they are going away or completely obscured.
The Last Week in AI podcast from Sept 8 seems to have gotten this wrong as well. Jeremie Harris rages that OpenAI implemented latent reasoning, ala the coconut paper, which could potentially actually obscure reasoning traces. But for the life of me, I do not know how he arrived at this conclusion and see no evidence that this has happened in Astra.
Interesting. I admit I was largely going on what Mr. Harris said on the episode when responding to several posts.
It can lead to hidden reasoning, if the looping allows it to stuff enough information outside visible CoT. Open AI demostrates such an ability by asking it to solve problems while thinking about something else entirely. All the other models are unable to do this except Astra. It doesn't have to be a substitute for CoT to cause monitorability issues.
If you ask it not to think about something that doesn't cause the pink elephant issue?
There's latent space thinking inside the model and then there's the thinking chain of thought words you see the model output. Of course the former is still happening even when you say 'don't think about it' but the latter can be controlled a great deal better with Astra.