Comment by profsummergig
5 days ago
The human mind cannot comprehend the capacity of massively multidimensional space.
Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...).
Now imagine 1,000 dimensions.
Imagine a box of balls. They have size, weight, colour, density… etc. These properties, each a measure, are dimensions and they are orthogonal to each other. Taken together are multi-dimensional.
Now take a set of words. They have "sizeness", "weightness", "colorness" and "densityness"...and "pythonness" and "haskellness" and even "adjectiveness" and "verbness" and so on and so forth...You can readily see that this can encode arbitrary patters (like language grammer or program syntax)
Training identifies these dimensions in the training data and links it with each word/token. Then given a stream of such tokens, each with its own set of dimensions (which can be huge), and LLM predicts the dimensions that the next token is most likely to have...
my take from all these is that is perhaps possible to teach similarity search to 5th graders. not an elementary teacher myself, but many math concepts are much easier to explain with allegories (visual and other) than with formalism. a good example perhaps is what Mamikon does in his visual explanations of calculus (https://calteches.library.caltech.edu/4007/1/Calculus.pdf)
formalisms come only after there is understanding of the phenomena and a specific need to communicate concepts across members of community - in a concise and commonly comprehensive way. my understanding is that education often fails to train understanding and jumps directly to formalisms. and this is right also for STEM schools, where it is presumed everyone has some innate inclination to the matter. but this presumption, in my experience as university teacher, is not entirely correct, and students are not universally prepared to express lack of understanding, and particularly when it is expected from them to rather indicate understanding.
it seems very weird, the concept, to reward the correct answer, while suppressing and discouraging the expression of misunderstanding or confusion. why is weird? because learning comes trough trial and error, and the error part is half of it. so we need to encourage people to express their errors or lack of understanding, so that we can then fill in the gaps and eventually stand on a common ground. sadly, many forums, HN included, does not cherish misunderstanding. our posts here mostly get upvoted whenever they cross a sort of threshold about the amount of truth in our writing, and not because of daring expressions of confusion.
This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.
Yes. It's worth pointing out that anything with n distinct parameters is just a point in n-dimensional space. We're so used to handling so many dimensions that nobody ever bats an eye until someone brings up the magic word "dimensions". It's quite intuitive actually.
The trivial example that comes to mind is the character customization sliders in many video games.
I think you accidentally a word, there. GP is talking about comprehending the capacity of massively multi-dimensional space.
Correct. I implied nothing about visualizing. I was talking about (information) capacity.
[dead]
I think the human mind can visualize anything, eg here are some visual descriptions of higher dimensional objects: https://gfody.substack.com/p/mini-hyper-golf
My aphantasia says your milage may vary on that.
On the other hand I deal with higher dimensions better than most, possibly because of my aphantasia.
I think it would be fair to say that at least a few people can do a small amount of 4d visualisation.
Agreed, I don't think anyone has a good grasp on, say, 64K D, but we can do a bit better than just 3.
I’ve heard people claiming that they can but I’ve never heard compelling evidence that they’re directly visualising 4D objects rather than 3D projections of them, or some symbolic representation (arrays of numbers etc.)
4 replies →
>The human mind cannot comprehend the capacity of massively multidimensional space.
That is why the scam works, because investors are humans...
Which scam sorry?
The scam that is based on the implicit claim that LLM is a path to AGI.
Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...
18 replies →