Comment by Workaccount2
2 months ago
Can you have a hybrid model that can do autoregression and diffusion? It doesn't seem like there is something that would fundamentally prevent this. A model with diffusion CoT for rapid "thought" generation, and then autoregression for the answer on the output.
You can absolutely do it, and I think it's a nice idea to try.