In the research literature this is generally called coconut after the first paper published on the idea. It has advantages and disadvantages. I agree it’s more efficient. But the lack of observability and transferability are real downsides that block its adoption.
Can you please elaborate? I hadn't found any sources and Google points to this post as the #1 use of "Neuralisee".
Switching the thinking from sampled coherent language tokens to raw logits not scored to correspond to any language.
In the research literature this is generally called coconut after the first paper published on the idea. It has advantages and disadvantages. I agree it’s more efficient. But the lack of observability and transferability are real downsides that block its adoption.
1 reply →
Correct spelling is "neuralese"
There is a computerphile video on this exact topic. https://www.youtube.com/watch?v=iuHddnIzKRA
They meant "neuralese".