Comment by abecedarius

> processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations

If the workings of those circuits are obvious to you, I'd really like to learn. Do you mean the level of analysis at https://transformer-circuits.pub/ ? (That looks like good work but not a deep understanding.)

Hofstadter referenced this back in the day as a promising beginning: https://en.wikipedia.org/wiki/Sparse_distributed_memory which sounds kind of similar in style to the embeddings you bring up.