Comment by jameshart
17 hours ago
Embedding spaces are all about analogy.
I’m confused why there seems to be a dismissal of the most basic ‘strange loop’ of the LLM - the fact that it’s evaluating a context to choose the next word, then reevaluating in a context where that word has been appended.
That always seemed to me like the essence of a Hofstadterish strange loop, so the emergence of Hofstadterish phenomena (self rep, etc) doesn’t seem surprising.
That's a really interesting point. I hadn't drawn that connection before.
And if anyone's reading this and hasn't read Hofstadter, you're making a mistake, it's utterly perspective-changing stuff. Well, it was for me, at least.
And don’t feel you need to go straight to GEB, either. I think his collection of Scientific American columns, Metamagical Themas, is also great and much more accessible.
You are guilt-tripping me for using my copy of Godel, Escher, Bach to raise my monitor before I got around to reading it.
It's a book that works on many levels.
3 replies →
I read it when I was about 9 or 10, I didn't understand any of it at all but I used to pick it up and look at the pictures of the Escher drawings, which were amazing.
Then, after I graduated from college, a friend of mine who was finishing his physics degree mentioned that he had just finished it and that it was life-changing. I read it again and deeply regretted having put it down for those 12 intervening years.
1 reply →