Comment by daemonk
4 days ago
Seems like the TLDR is that the neural network is not randomly placing things together due to constraint, it is actually finding some commonality from the input dataset. We just need to spend more effort trying to trace it.
Perhaps our anthropomorphic understanding of data is biasing what objective features there actually are in the dataset.
I wonder if we can study the graph similarities of various models, do we see a convergence of structures that encapsulate certain concepts. And would that convergence be due to just how humans have narrowly defined the concept in our language, or is there something inherent to the concept that allows it to converge. If we compare the same concept in different languages (ie. chinese vs english), does this convergence disappear?
Why would it 'randomly' place things anyway. If there is ever a space constraint the most space-saving way is to group into highly generalizable concepts.
As a lay-person I'm really confused by this. I thought it was obvious that these networks were already picking up on shared features such as 'being V-shaped'. Isn't that why older and smaller models were getting confused between objects? What am I missing.
Yeah very confusing why someone would think cars and cats are not similar. They both have two shiny eyes and a nose!
Who knows what they were thinking before, but this all sounds very good.
If you view LLMs as lossy data compressors, this exactly what you'd expect.
If your view is anthropomorphic, i.e. LLMs internal form being structurally equivalent to ours at a basic level, this is also exactly what you'd expect.