Comment by megaloblasto
3 hours ago
I think provable conclusions and building an intuition are different things to be fair. In my opinion it is entirely possible to build intuition about seemingly impossible to understand topics like infinitesimal areas, infinite limits, function mappings, and high dimensional spaces. For example, 3D spaces are easy to intuit if you can visualize vectors in your head. Some people claim it is impossible to understand dimensions higher than 3 but that's obviously not true because you could assign a color to each vector to visualize the next dimension. You could then assign a width to visualize a 5th dimension, and an arrow shape to signify the next, and so on and so on. That's more of a creative act that can start to build at least a visual understanding in you head of these higher dimension spaces. That's a way to start to intuit about those things.
Now with llms we need way more that 6 dimensions so we can start thinking of assigning matrices to each 3d point for example. That allows us to increase the dimension from 3 to 3 + whatever the matrix dimension is.
We can visualize the matrices instead of having numbers as having colors for each entry, so they can be a sort of cube with each vowel being a different color.
Now you can start to visually intuit about how these massively high dimensional spaces can be formed of these colored matrices that can react to some input training data.
That's a start of an idea for intuiting things that might seem impossible to have an intuition about. I think visualization is a great way to start.
i read all that i still dont see what your intuition is about how llm work.
visualizign 100 dim matrix will not tell you how llm work. so what you even talking about.
Yes, if it cannot be measured it doesn’t exist no matter how much it would make everyone feel more comfortable if we could intuitively understand these systems.
Visualizing large dimensional matrices is a way of starting to develop intuition about llms. If you can't visualize high dimensional spaces, and you haven't studied llms, you're not going to have any intuition about how they work.
Depending on your knowledge of math, I recommend starting with linear algebra, building an understanding of the equations and try to visualize more and more complex systems, then study llms to see how you can apply your linear algebra intuition to your understanding of llms.
VTK is a great toolkit for visualizing complex systems. 3 blue one brown on YouTube has other visuals that might help you.
It takes time but it's possible. Good luck.